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Why GAO Did This Study 

The Department of Homeland 
Security (DHS) has established a 
program — the U.S. Visitor and 
Immigrant Status Indicator 
Technology (US-VISIT)— to collect, 
maintain, and share information, 
including biometric identifiers, on 
selected foreign nationals entering 
and exiting the United States. 
US- VISIT uses these identifiers 
(digital fingerscans and 
photographs) to screen persons 
against watch lists and to verify 
that a visitor is the person who was 
issued a visa or other travel 
document. Visitors are also to 
confirm their departure by having 
their visas or passports scanned 
and undergoing fingerscanning at 
selected air and sea ports of entry 
(POE). 

GAO has made many 
recommendations to improve the 
program, all of which DHS has 
agreed to implement. GAO was 
asked to report on DHS's progress 
in responding to 18 of these 
recommendations. 



What GAO Recommends 



GAO is closing its existing 
recommendation related to DHS's 
assessment of Increment 2B and 
recommending that DHS explore 
alternative means to fully assess 
the impact of US-VISIT entry 
capabilities on land POEs. In its 
comments on a draft of this report, 
DHS stated that it agreed with 
many areas of the report and 
disagreed with others. It also 
concurred with the need to quickly 
implement GAO's open 
recommendations. 

www.gao.gov/cgi-bin/getrpt7GAO-06-296. 

To view the full product, including the scope 
and methodology, click on the link above. 
For more information, contact Randolph C. 
Hite at (202) 512-3439 or hiter@gao.gov. 



What GAO Found 

The current status of DHS's implementation of the 18 recommendations is 
mixed, but progress in critical areas has been slow. DHS has implemented 2 
of the recommendations: it defined program staff positions, roles, and 
responsibilities, and it hired an independent verification and validation 
contractor. It has also taken steps to implement the other recommendations, 
partially completing 11 and beginning to implement another 5. 

• In September 2003, GAO reported that the program had not assessed the 
costs and benefits of Increment 1 (which provides entry capabilities to 
air and sea POEs) and recommended that the program determine 
whether proposed increments will produce mission value commensurate 
with cost. In the latest cost-benefit analysis, dated June 23, 2005, the 
program identified potential costs and benefits for three alternatives for 
an air and sea exit solution. However, the analysis does not meet key 
Office of Management and Budget criteria; for example, it does not 
include a complete uncertainty analysis, which helps to provide decision 
makers with perspective on the potential variability of the cost and 
benefit estimates should circumstances change. 

• GAO reported in May 2004 and February 2005 that system testing was 
not based on well-defined test plans and recommended that before 
testing begins, the program develop and approve test plans meeting 
certain criteria. However, although the latest test plan did cover many 
required areas (such as the tests to be performed), it did not adequately 
trace between test cases and the requirements to be verified by testing. 
Without complete and traceable test plans, the risk is increased that the 
deployed system will not perform as intended. 

• In May 2004, GAO reported that the program had not assessed its 
workforce and facility needs for Increment 2B (which extends entry 
capabilities to the 50 busiest land POEs) and recommended that it do so. 
Since then, the program evaluated the processing times to issue and 
process entry/exit forms at 3 of the 50 busiest POEs and concluded that 
the results showed that no additional staff and only minor facilities 
modifications were required. However, the scope of the evaluation was 
limited. Since then, DHS has deployed and implemented Increment 2B 
capabilities to these 50 POEs, making the collection of predeployment 
baseline data for these sites impractical. Nonetheless, other alternatives, 
such as surveying site officials about the increment's impacts, have yet 
to be explored. Until they are, the program may not be able to accurately 
project resource needs or make any needed modifications to achieve its 
goals of minimizing US-VISIT's impact on POE operations, which was 
the impetus for GAO's recommendation. 

DHS attributed the pace of progress to competing demands on time and 
resources. The longer that US- VISIT takes to implement the 
recommendations, the greater the risk that the program will not meet its 
stated goals on time and within budget. 
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A G A O 

^^^^^^^^^Accountability * Integrity * Reliability 

United States Government Accountability Office 
Washington, D.C. 20548 



February 14, 2006 
Congressional Requesters: 

The U.S. Visitor and Immigrant Status Indicator Technology (US-VISIT) is a 
multibillion-dollar program of the Department of Homeland Security (DHS) 
that is intended to record the entry into and exit from the United States of 
selected individuals, verify their identity, and confirm their compliance 
with the terms of their admission into and stay in the United States. The 
goals of the program are to (1) enhance the security of our citizens and 
visitors, (2) facilitate legitimate travel and trade, (3) ensure the integrity of 
the U.S. immigration system, and (4) protect the privacy of our visitors. 

Since fiscal year 2002, DHS has been legislatively directed to submit annual 
expenditure plans for the program, and we have been directed to review 
these plans and issue reports. These reports have, among other things, 
identified risks that face the department in delivering promised program 
capabilities and benefits on time and within cost. 1 For example, we 
reported that the program office did not have the human capital and 
acquisition process discipline needed to effectively manage the program. 
Because of the number and severity of program management challenges 
that we identified, we concluded that the program was risky. 

To address program risks, our reports have included 18 recommendations 
in such areas as system acquisition process controls, economic 
justification, human capital management, cost estimating, and test 
management, all of which DHS has agreed to implement. 2 Because of your 
continued interest in ensuring that DHS is taking the necessary actions to 
successfully implement US- VISIT, you asked us to determine the progress 



'Our previous reports regarding US-VISIT's expenditure plans, which include 
recommendations, were published in GAO, Homeland Security: Some Progress Made, but 
Many Challenges Remain on U.S. Visitor and Immigrant Status Indicator Technology 
Program, GAO-05-202 (Washington, D.C: Feb. 23, 2005); Homeland Security: First Phase of 
Visitor and Immigration Status Program Operating, but Improvements Needed, 
GAO-04-586 (Washington, D.C: May 11, 2004); Homeland Security: Risks Facing Key 
Border and Transportation Security Program Need to Be Addressed, GAO-03-1083 
(Washington, D.C: Sept. 19, 2003); and Information Technology: Homeland Security Needs 
to Improve Entry Exit System Expenditure Planning, GAO-03-563 (Washington, D.C: June 
9,2003). 

2 Our reports included 24 recommendations, of which 6 related specifically to the contents of 
the expenditure plan. Those 6 are not included in the scope of this report, but they will be 
included in the scope of our fiscal year 2006 expenditure plan review. 
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being made in implementing these recommendations. To achieve this 
objective, we analyzed program plans, reports, and system documentation 
relative to the intent of each of our recommendations, and we interviewed 
appropriate DHS and program officials. (Further details on our objective, 
scope, and methodology are provided in app. I.) Our work was performed 
from August 2005 through December 2005 in accordance with generally 
accepted government auditing standards. 



RGSllltS in Brief ^ ne curren t status of DHS's implementation of the 18 recommendations is 

mixed, but progress in critical areas has been slow. DHS has implemented 2 
of the recommendations: it defined program staff positions, roles, and 
responsibilities, and it hired an independent verification and validation 
contractor. It has also taken steps to implement the other 
recommendations, partially completing 11 and beginning to implement 
another 5. However, although considerable time has passed since the 
recommendations were made, key actions have not yet been taken in such 
critical areas as (1) assessing security risks and planning for cost-effective 
controls to address the risks, (2) determining — before US- VISIT increments 
are deployed — whether each increment will produce mission value 
commensurate with cost and risk, and (3) ensuring that each increment is 
adequately tested. Of the 11 recommendations that are partially 
implemented, 7 are about 2 years old, and 4 are about 10 to 19 months old. 
Of the 5 that are in progress, 3 are about 10 months old. 3 According to the 
Program Director, the pace of progress is attributable to competing 
demands on time and resources. The longer that US- VISIT takes to 
implement the recommendations, the greater the risk that the program will 
not meet its stated goals on time and within budget. 

DHS provided written comments on a draft of this report. In its comments, 
the department stated that it agreed with many areas of the report and that 
our recommendations had made US- VISIT a stronger program. Further, the 
department stated that while it disagreed with certain areas of the report, it 
nevertheless concurred with the need to implement our open 
recommendations with all due speed and diligence. One area of 
disagreement was regarding the program's ability to thoroughly assess the 



3 We considered a recommendation (1) completely implemented when documentation 
demonstrated that it had been fully addressed, (2) partially implemented when 
documentation indicated that actions were under way to implement it, and (3) in progress 
when documentation indicated that actions had been initiated to implement it. 



Page 2 



GAO-06-296 US-VISIT Recommendations 



impact of US-VISIT entry capabilities on the 50 busiest land port of entry 
(POE) facilities and staffing levels, an assessment that we called for in our 
recommendation. In particular, DHS stated that since US- VISIT was 
operational at these POEs, the collection of predeployment baseline 
performance data was no longer practical. In light of these comments, we 
are making a new recommendation to the Secretary of DHS that recognizes 
these facts and circumstances and that replaces the open recommendation 
discussed in this report. This recommendation provides for the department 
to explore alternative means of assessing the impact of US-VISIT entry 
capabilities on land POE facilities and staffing levels. All of DHS's 
comments, along with our responses, are discussed in detail in the Agency 
Comments and Our Evaluation section of this report. The comments are 
also reprinted in their entirety in appendix II. 



Background US- VISIT is a governmentwide program intended to enhance the security of 

U.S. citizens and visitors, facilitate legitimate travel and trade, ensure the 
integrity of the U.S. immigration system, and protect the privacy of our 
visitors. Its scope includes the pre-entry, entry, status, and exit of hundreds 
of millions of foreign national travelers who enter and leave the United 
States at over 300 air, sea, and land POEs, and the provision of new 
analytical capabilities across the overall process. 

To achieve its goals, US- VISIT uses biometric information (digital 
fingerscans and photographs) to verify identity. 4 In many cases, the 
US-VISIT process begins overseas at U.S. consular offices, which collect 
biometric information from applicants for visas and check this information 
against a database of known criminals and suspected terrorists. When a 
visitor arrives at a POE, the biometric information is used to verify that the 
visitor is the person who was issued the visa. In addition, at certain sites, 
visitors are required to confirm their departure by undergoing US-VISIT 
exit procedures — that is, having their visas or passports scanned and 
undergoing fingerscanning. The exit confirmation is added to the visitor's 
travel records to demonstrate compliance with the terms of admission to 
the United States. (App. Ill provides a detailed description of the pre-entry, 
entry, status, exit, and analysis processes.) 



4 Biometric comparison is a means of identifying a person by biological features unique to 
that individual. 
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Key US-VISIT functions include 



• collecting, maintaining, and sharing information on certain foreign 
nationals who enter and exit the United States; 

• identifying foreign nationals who (1) have overstayed or violated the 
terms of their admission; (2) may be eligible to receive, extend, or adjust 
their immigration status; or (3) should be apprehended or detained by 
law enforcement officials; 

• detecting fraudulent travel documents, verifying traveler identity, and 
determining traveler admissibility through the use of biometrics; and 

• facilitating information sharing and coordination within the immigration 
and border management community 

In July 2003, DHS established a program office with responsibility for 
managing the acquisition, deployment, operation, and sustainment of the 
US-VISIT system and its associated supporting people (e.g., Customs and 
Border Protection (CBP) officers), processes (e.g., entry/exit policies and 
procedures), and facilities (e.g., inspection booths and lanes), in 
coordination with its stakeholders (CBP and the Department of State). 

As of October 2005, about $1.4 billion has been appropriated for the 
program, and, according to program officials, about $962 million has been 
obligated. 



DHS plans to deliver US- VISIT capability in four increments, with 
Increments 1 through 3 being interim, or temporary, solutions that fulfill 
legislative mandates to deploy an entry/exit system, and Increment 4 being 
the implementation of a long-term vision that is to incorporate improved 
business processes, new technology, and information sharing to create an 
integrated border management system for the future. In Increments 1 
through 3, the program is building interfaces among existing ("legacy") 
systems; enhancing the capabilities of these systems; and deploying these 
capabilities to air, sea, and land POEs. These increments are to be largely 
acquired and implemented through existing system contracts and task 
orders. 



Acquisition and 
Implementation Strategy: A 
Brief Description 
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In May 2004, DHS awarded an indefinite-delivery/indefinite-quantity 5 prime 
contract to Accenture and its partners. According to the contract, the prime 
contractor will help support the integration and consolidation of processes, 
functionality, and data, and it will develop a strategy to build on the 
technology and capabilities already available to produce the strategic 
solution, while also assisting the program office in leveraging existing 
systems and contractors in deploying the interim solutions. 



US-VISIT Is Being 
Implemented in Four 
Increments 



Increment 1 concentrates on establishing capabilities at air and sea POEs. 
It is divided into two parts — 1 and IB. 

• Increment 1 (air and sea entry) includes the electronic capture and 
matching of biographic and biometric information (two digital index 
fingerscans and a digital photograph) for selected foreign nationals, 
including those from visa waiver countries. 6 Increment 1 was deployed 
on January 5, 2004, for individuals requiring a nonimmigrant visa to 
enter the United States, through the modification of pre-existing 
systems. 7 These modifications accommodated the collection and 
maintenance of additional data fields and established interfaces 
required to share data among DHS systems in support of entry 
processing at 115 airports and 14 seaports. 



Increment IB (air and sea exit) involves the testing of exit devices to 
collect biometric exit data for select foreign nationals at 11 airports and 
seaports. Three exit alternatives were pilot tested: 

• Kiosk — A self-service device (which includes a touch-screen 
interface, document scanner, finger scanner, digital camera, and 
receipt printer) that captures a digital photograph and fingerprint and 
prints out an encoded receipt. 



5 An indefinite-delivery/indefmite-quantity contract provides for an indefinite quantity, within 
stated limits, of supplies or services during a fixed period of time. The government 
schedules deliveries or performance by placing orders with the contractor. 

6 The Visa Waiver Program permits foreign nationals from designated countries to apply for 
admission to the United States for a maximum of 90 days as nonimmigrant visitors for 
business or pleasure. 

7 On September 30, 2004, US-VISIT expanded biometric entry procedures to include 
individuals from visa waiver countries applying for admission. 
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• Mobile device — A hand-held device that is operated by a workstation 
attendant; 8 it includes a document scanner, finger scanner, digital 
camera, and receipt printer and is used to capture a digital 
photograph and fingerprint. 

• Validator — A hand-held device that is used to capture a digital 
photograph and fingerprint, which are then matched to the 
photograph and fingerprint captured via the kiosk and encoded in the 
receipt. 

Increment 2 focuses primarily on extending US-VISIT to land POEs. It is 
divided into three parts — 2A, 2B, and 2C. 

• Increment 2A (air, sea, and land) includes the capability to biometrically 
compare and authenticate valid machine-readable visas and other travel 
and entry documents issued by State and DHS to foreign nationals at all 
POEs. Increment 2A was deployed on October 23, 2005, according to 
program officials. It also includes the deployment by October 26, 2006, 
of technology to read biometrically enabled passports from visa waiver 
countries. 

• Increment 2B (land entry) redesigns the Increment 1 entry solution and 
expands it to the 50 busiest land POEs. The process for issuing Form 
I-94 9 was redesigned to enable the electronic capture of biographic, 
biometric (unless the traveler is exempt), and related travel 
documentation for arriving travelers. This increment was deployed to 
the busiest 50 U.S. land border POEs as of December 29, 2004. Before 
Increment 2B, all information on the Form I-94s was handwritten. The 
redesigned systems electronically capture the biographic data included 
in the travel document. In some cases, the form is completed by CBP 
officers, who enter the data electronically and then print the form. 



Workstation attendants assist travelers in using the kiosk. 

9 Form I-94s are used to record a foreign national's entry into the United States. The form has 
two parts — arrival and departure — and each part contains a unique number for the purposes 
of recording and matching the arrival and departure records of nonimmigrants. 
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• Increment 2C is to provide the capability to automatically, passively, and 
remotely record the entry and exit of covered individuals using radio 
frequency (RF) technology tags at primary inspection and exit lanes. 10 
An RF tag that includes a unique ID number is to be embedded in each 
Form 1-94, thus associating a unique number with a record in the 
US-VISIT system for the person holding that Form 1-94. In August 2005, 
the program office deployed the technology to five border crossings 
(three POEs) to verify the feasibility of using passive RF technology to 
record traveler entries and exits via a unique ID number embedded in 
the CBP Form 1-94. The results of this demonstration are to be reported 
in February 2006. 

Increment 3 extended Increment 2B (land entry) capabilities to 104 land 
POEs; this increment was essentially completed as of December 19, 2005. 11 

Increment 4 is the strategic US- VISIT program capability, which program 
officials stated will likely consist of a further series of incremental releases 
or mission capability enhancements that will support business outcomes. 
The program reports that it has worked with its prime contractor and 
partners to develop this overall vision for the immigration and border 
management enterprise. 

Increments 1 through 3 include the interfacing and integration of existing 
systems and, with Increment 2C, the creation of a new system, the 
Automated Identification Management System (AIDMS). The three main 
existing systems are as follows: 

• The Arrival Departure Information System (ADIS) stores 

• noncitizen traveler arrival and departure data received from air and 
sea carrier manifests, 

• arrival data captured by CBP officers at air and sea POEs, 



10 RF technology relies on proximity cards and card readers. RF devices read the information 
contained on the card when the card is passed near the device and can also be used to verify 
the identity of the cardholder. 

u At one POE, these capabilities were deployed by December 19, 2005, but were not fully 
operational until January 7, 2006, because of a telephone company strike that prevented the 
installation of a T-l line. 
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• Form 1-94 issuance data captured by CBP officers at Increment 2B 
land POEs, 

• departure information captured at US- VISIT biometric departure 
pilot (air and sea) locations, 

• pedestrian arrival information and pedestrian and vehicle departure 
information captured at Increment 2C POE locations, and 

• status update information provided by the Student and Exchange 
Visitor Information System (SEVIS) and the Computer Linked 
Application Information Management System (CLAIMS 3) (described 
below). 

ADIS provides record matching, query, and reporting functions. 

• The passenger processing component of the Treasury Enforcement 
Communications System (TECS) includes two systems: Advance 
Passenger Information System (APIS), a system that captures arrival 
and departure manifest information provided by air and sea carriers, 
and the Interagency Border Inspection System, a system that maintains 
lookout data and interfaces with other agencies' databases. CBP officers 
use these data as part of the admission process. The results of the 
admission decision are recorded in TECS and ADIS. 

• The Automated Biometric Identification System (IDENT) collects and 
stores biometric data on foreign visitors. 

US-VISIT also exchanges biographic information with other DHS systems, 
including SEVIS and CLAIMS 3. These two systems contain information on 
foreign students and foreign nationals who request benefits, such as a 
change of status or extension of stay. 

Some of the systems previously described, such as IDENT and the new 
AIDMS, are managed by the program office, while some systems are 
managed by other organizational entities within DHS. For example, TECS 
is managed by CBP, SEVIS is managed by Immigration and Customs 
Enforcement, CLAIMS 3 is under United States Citizenship and 
Immigration Services, and ADIS is jointly managed by CBP and US-VISIT. 

US-VISIT also interfaces with other, non-DHS systems for relevant 
purposes, including watch list updates and checks to determine whether a 
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visa applicant has previously applied for a visa or currently has a valid U.S. 
visa. In particular, US- VISIT receives biographic and biometric information 
from State's Consular Consolidated Database as part of the visa application 
process, and returns fingerscan information and watch list changes. 



Program Management Roles 
and Responsibilities 



The US- VISIT program office structure includes nine component offices. 
Each of the program offices includes a director and subordinate 
organizational units, as established by the director. The responsibilities for 
each office are stated below. Figure 1 shows the program office structure, 
including its nine offices. 



Figure 1 : US-VISIT Program Office Structure 
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The roles and responsibilities for each of the nine offices include the 
following: 

• Chief Strategist is responsible for developing and maintaining the 
strategic vision, strategic documentation, transition plan, and business 
case. 

• Budget and Financial Management is responsible for establishing the 
program's costs estimates; analysis; and expenditure management 
policies, processes, and procedures that are required to implement and 
support the program by ensuring proper fiscal planning and execution 
of the budget and expenditures. 

• Mission Operations Management is responsible for developing 
business and operational requirements based on strategic direction 
provided by the Office of the Chief Strategist. 

• Outreach Management is responsible for enhancing awareness of 
US-VISIT requirements among foreign nationals, key domestic 
audiences, and internal stakeholders by coordinating outreach to media, 
third parties, key influencers, Members of Congress, and the traveling 
public. 

• Information Technology Management is responsible for developing 
technical requirements based on strategic direction provided by the 
Office of the Chief Strategist and business requirements developed by 
the Office of Mission Operations Management. 

• Implementation Management is responsible for developing accurate, 
measurable schedules and cost estimates for the delivery of mission 
systems and capabilities. 

• Acquisition and Program Management is responsible for establishing 
and managing the execution of program acquisition and management 
policies, plans, processes, and procedures. 

• Administration and Training is responsible for developing and 
administering a human capital plan that includes recruiting, hiring, 
training, and retaining a diverse workforce with the competencies 
necessary to accomplish the mission. 
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• Facilities and Engineering Management is responsible for establishing 
facilities and environmental policies, procedures, processes, and 
guidance required to implement and support the program office. 



Our Prior Work Has m response to legislative mandate, we have issued four reports on DHS's 

Resulted in Several annual expenditure plans for US- VISIT. 12 Our reports have, among other 

Recommendations things, assessed whether the plans satisfied the legislative conditions and 

provided observations on the plans and DHS's program management. As a 
result of our assessments, we made 24 recommendations aimed at 
improving both plans and program management, all of which DHS has 
agreed to implement. Of these 24 recommendations, 18 address risks 
stemming from program management. 13 



The Status of DHS's 
Implementation of Our 
Recommendations Is 
Mixed 



The current status of DHS's implementation of our 18 recommendations on 
program risks is mixed, but progress in critical areas has been slow. For 
example, over 2 years have passed, and the program office has yet to 
develop a security plan consistent with federal guidance or to economically 
justify its investment in system increments. According to the Program 
Director, the pace of progress is attributable to competing demands on 
time and resources. 



DHS agreed to implement all 18 recommendations. Of these 18, DHS has 
completely implemented 2, has partially implemented 11, and is in the 
process of implementing another 5. Of the 11 that are partially 
implemented, 7 are about 2 years old, and 4 are about 10 to 19 months old. 
Of the 5 that are in progress, 3 are about 10 months old. 

These 18 recommendations are aimed at strengthening the program's 
management effectiveness. The longer that the program takes to 
implement the recommendations, the greater the risk that the program will 
not meet its goals on time and within budget. 



12 GAO-05-202, GAO-04-586, GAO-03-1083, and GAO-03-563. 

13 As previously mentioned, the remaining 6 recommendations related specifically to the 
contents of the expenditure plans and are not reported on in this report; their status will be 
included in the scope of our fiscal year 2006 expenditure plan review. 
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Figure 2 provides an overview of the extent to which each recommendation 
has been implemented. The figure is followed by sections providing details 
on each recommendation and our assessment of its implementation status. 
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Figure 2: DHS's Progress toward Implementing GAO's 18 Recommendations 
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Develop and begin implementing a system security plan and privacy impact assessment. 




€ 




About 27 months 

Develop and implement a plan for satisfying key acquisition management controls, including acquisition planning, 
solicitation, requirements development and management, project management, contract tracking and oversight, 
evaluation, and transition to support, and implement the controls in accordance with Software Engineering Institute 01 
guidance. 








Determine whether proposed US-VISIT increments will produce mission value commensurate with cost and risks. 




€ 




Clarify the operational context in which US-VISIT is to operate. 






o 


Ensure that human capital and financial resources are provided to establish a fully functional and effective program 
office. 




o 




Define program office positions, roles, and responsibilities. 


• 






Develop and implement a human capital strategy for the program office that provides for staffing positions with 
individuals who have the appropriate knowledge, skills, and abilities. 




€ 




Define performance standards for each increment that are measurable and reflect the limitations imposed by relying 
on existing systems. 




C 




Develop and implement a risk management plan and report all high risks and their status to the executive body on a 
regular basis. 




€ 




About 19 months 

Develop and approve test plans before testing begins. These test plans should (1) specify the test environment; 
(2) describe each test to be performed, including test controls, inputs, and expected outputs; (3) define the test 
procedures to be followed in conducting the tests; and (4) provide traceability between test cases and the require- 
ments to be verified by the testing. 




C 




Assess the full impact of Increment 2B on land ports of entry workforce levels and facilities, including performing 
appropriate modeling exercises. 




€ 




Implement effective configuration management practices, including establishing a change control board to manage 
and oversee system changes. 






o 


Ensure the independence of the independent verification and validation contractor. 


• 






Develop a plan, including explicit tasks and milestones, for implementing all our open recommendations and 
periodically report to the DHS Secretary and Under Secretary on progress in implementing this plan, and report on 
this progress, including reasons for delays, in all future expenditure plans. 




€ 




About 10 months 

Follow effective practices for estimating the costs of future increments. 






o 


Reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides for adequate 
evaluation of alternative solutions and better ensures that the exit solution selected is in the best interest of the 
program. 








Develop and implement processes for managing the capacity of the system. 






o 


Make understanding the relationships and dependencies between the US-VISIT and Automated Commercial 
Environment 6 programs a priority matter, and report periodically to the Under Secretary on progress in doing so. 






o 



Sources: US-VISIT, GAO (analysis), Nova Development Corp. (images). 



a A recommendation is completely implemented when documentation demonstrated that it had been 
fully addressed. 
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b A recommendation is partially implemented when documentation indicated that actions were under 
way to implement it. 

°A recommendation is in progress when documentation indicated that actions had been initiated to 
implement it. 

"Carnegie Mellon University Software Engineering Institute, Software Acquisition Capability Maturity 
Model, Version 1.03 (March 2002). 

e Automated Commercial Environment is a new trade processing system planned to support the 
movement of legitimate imports and exports and to strengthen border security. 



Development and 
Implementation of a 
Security Plan and 
Performance of a Privacy 
Impact Assessment Are 
Partially Complete 



In June 2003, 14 we reported that the Immigration and Naturalization 
Service 15 had not developed a security plan and performed a privacy impact 
assessment for the entry exit program (as US-VISIT was then known). A 
security plan and privacy impact assessment are important to 
understanding system requirements and ensuring that the proper 
safeguards are in place to protect system data and resources. System 
acquisition best practices and federal guidance advocate understanding 
and defining security and privacy requirements both early and continuously 
in a system's life cycle, and effectively planning for their satisfaction. 
Accordingly, we recommended that DHS do the following: 



Develop and begin implementing a system security plan, and perform a 
privacy impact assessment and use the results of the analysis in near-term 
and subsequent system acquisition decision making. 



14 GAO-03-563. 

I5 In March 2003, the Immigration and Naturalization Service was subsumed within DHS, 
and, in April 2003, the entry exit program became known as US-VISIT. 
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Security Plan Since we made the system security plan recommendation about 2V2 years 

ago, its implementation has been slow. For example, we reported in 
September 2003 and again in May 2004 that the program office had not 
developed a security plan. In February 2005, we reported that the program 
office had developed a security plan, dated September 2004, and that this 
plan was generally consistent with federal guidance. 16 That is, the plan 
provided an overview of system security requirements, described the 
controls in place or planned for meeting those requirements, referred to the 
applicable documents that prescribe the roles and responsibilities for 
managing the US- VISIT component systems, and addressed security 
awareness and training. However, the program office had not conducted a 
risk assessment or included in the plan when an assessment would be 
completed. According to guidance from the Office of Management and 
Budget (OMB), the security plan should describe the methodology that is 
used to identify system threats and vulnerabilities and to assess risks, and 
it should include the date the risk assessment was completed. 

According to program officials, they completed a programwide risk 
assessment in December 2005, but have yet to provide a copy of the 
assessment to us. Therefore, we cannot confirm that the assessment has 
been done, and done properly. The absence of a risk assessment and a 
security plan that reflects this assessment is a significant program 
weakness. Risk assessments are critical to establishing effective security 
controls because they provide the basis for establishing appropriate 
policies and selecting cost-effective controls to implement these policies. 
Without such an assessment, US- VISIT does not have adequate assurance 
that it knows the risks associated with the program and thus whether it has 
implemented effective controls to address them. 

Notwithstanding these limitations in the security plan, the program office 
has begun to implement aspects of its September 2004 security plan. For 
example, the Information Systems Security Manager told us that a security 
awareness program is established and key personnel have attended 
security training. 



16 OMB, Security of Federal Automated Information Resources, Circular A-130, Revised 
(Transmittal Memorandum No. 4), Appendix III (Washington, D.C.: Nov. 28, 2000); and 
National Institute of Standards and Technology, Guide for Developing Security Plans for 
Information Technology Systems, Special Publication 800-18 (December 1998). 
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Privacy Impact Assessment Since June 2003, US- VISIT has also developed and periodically updated a 

privacy impact assessment. An initial impact assessment was issued in 
January 2004, and a revised assessment was issued in September 2004. 17 A 
more recent assessment, dated July 2005, reflects changes related to 
Increments IB and 2C. Each of these assessments is generally consistent 
with OMB guidance. 18 That is, each of the assessments addressed most 
OMB requirements, including the impact that the system will have on 
individual privacy, the privacy consequences of collecting the information, 
and alternatives considered to collect and handle information. The most 
recent impact assessment, for example, states that three alternatives were 
considered for Increment IB — the kiosk, the mobile device, and the 
validator (a combination of the two) — and discusses proposals to mitigate 
the privacy risks of all three, such as by limiting the duration of data 
retention on the exit devices and using encryption. 

However, OMB guidance also requires that privacy impact assessments 
developed for systems under development address privacy in relevant 
system documentation, including statements of need, functional 
requirements documents, and cost-benefit analyses. As we reported about 
previous privacy impact assessments, privacy is only partially addressed in 
system documentation. For example, the Increment IB cost-benefit 
analysis assesses the privacy risk associated with each exit alternative, and 
the Increment 2C business requirements state that all solutions are to be 
compliant with privacy laws and regulations and adhere to US- VISIT 
privacy policy. However, we did not find privacy in the Increment IB 
business requirements or the Increment 2C functional requirements. 
Program officials, including the US- VISIT Privacy Officer, acknowledged 
that privacy is not included in the system documentation, but stated that 
privacy is considered in the development of the documentation and that the 
privacy office reviews key system documentation at relevant times during 
the system development life cycle. Nevertheless, we did not find evidence 
of privacy being addressed in the system documentation, and program 
officials acknowledged that it was not included. 



"The initial assessment was updated in September 2004 to reflect the inclusion of Visa 
Waiver Program travelers in US-VISIT, the expansion of US-VISIT to the 50 busiest land 
border POEs (Increment 2B), and changes in the business processes used by DHS to share 
information with federal law enforcement agencies. The assessment was again updated in 
June 2005 to include the live test to read biometrically enabled travel documents (Increment 
2A). 

18 OMB, Guidance for Implementing the Privacy Provisions of the E-Government Act of 
2002, OMB M-03-22 (Sept. 26, 2003). 
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Until the program performs a risk assessment and fully implements a 
security plan that reflects this assessment, it cannot adequately ensure that 
US- VISIT is cost-effectively safeguarding assets and data. Moreover, 
without reflecting privacy in system documentation, it cannot adequately 
ensure that privacy needs are being fully addressed. 



We reported in September 2003 that the program office had not defined 
key acquisition management controls to support the acquisition of 
US-VISIT, and therefore its efforts to acquire, deploy, operate, and maintain 
system capabilities were at risk of not satisfying system requirements and 
of not meeting benefit expectations on time and within budget. 

The Capability Maturity Model-Integration® (CMMI) developed by 
Carnegie Mellon University's Software Engineering Institute (SEI) 
explicitly defines process management controls that are recognized 
hallmarks of successful organizations and that, if implemented effectively, 
can greatly increase the chances of successfully acquiring 
software-intensive systems. 20 SEFs CMMI model uses capability levels to 
assess process maturity. 21 Because establishing the basic acquisition 
process capabilities, according to SEI, can take on average about 19 
months, we recognized the importance of starting early to build effective 
acquisition management capabilities by recommending that DHS do the 
following: 



Develop and implement a plan for satisfying key acquisition management 
controls, including acquisition planning, solicitation, requirements 
management, program management, contract tracking and oversight, 
evaluation, and transition to support, and implement the controls in 
accordance with SEI guidance. 



Development and 
Implementation of Key 
Acquisition Controls Are 
Partially Complete 



19 GAO-03-1083. 

20 Carnegie Mellon University Software Engineering Institute, Capability Maturity Model 
Integration, Systems Engineering Integrated Product and Process Development, 
Continuous Representation, version 1.1 (March 2002). 

21 When we made our original recommendation, we referred to an earlier SEI model, the 
Software Acquisition Capability Maturity Model. However, SEI is transitioning to an 
integrated model, and the program office is using the CMMI model for its improvement 
program. 
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The program office has recently taken foundational steps to establish key 
acquisition management controls. For example, it has developed a process 
improvement plan, dated May 16, 2005 (about 20 months after our 
recommendation), to define and implement these controls. As part of its 
improvement program, the program office is implementing a governance 
structure for overseeing improvement activities, consisting of three groups: 
a Management Steering Group, an Enterprise Process Group, and Process 
Action Teams. Specific roles for each of these groups are described below. 

• The Management Steering Group is to provide policy and procedural 
guidance and to oversee the entire improvement program. The steering 
group is chaired by the US- VISIT Director, with the Deputy Director and 
the functional office directors serving as core members. 

• The Enterprise Process Group is to provide planning, management, and 
operational guidance in day-to-day process improvement activities. The 
group is chaired by the process improvement leader and is composed of 
individuals from each functional office. 

• Process Action Teams are to provide specific process documentation 
and to provide implementation support and training services. These 
teams are to be active as long as a particular process improvement 
initiative is under way. To date, the program office has chartered five 
process teams — configuration management, cost analysis, process 
development, communications, and policy. 

In addition, the program office has recently completed a self-assessment of 
its acquisition process maturity, and it plans to use the assessment results 
to establish a baseline of its acquisition process maturity for improvement. 
According to program officials, the assessment included 13 key process 
areas that are generally consistent with the process areas cited in our 
recommendation. The program has ranked these 13 process areas 
according to their priority, and, for initial implementation, it plans to focus 
on the following 6: 22 

• Configuration management. Establishing and maintaining the integrity 
of the products throughout their life cycle. 



22 The 7 remaining process areas are supplier agreement management, measurement and 
analysis, solicitation and contract monitoring, transition to operations and support, 
organizational training, organizational process focus, and organizational process definition. 
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• Process and product quality assurance. Taking actions to provide 
management with objective insight into the quality of products and 
processes. 

• Project monitoring and control. Tracking the project's progress so that 
appropriate corrective actions can be taken when performance deviates 
significantly from plans. 

• Project planning. Establishing and maintaining plans for work 
activities. 

• Requirements management. Managing the requirements and ensuring a 
common understanding of the requirements between the customer and 
the product developers. 

• Risk management. Identifying potential problems before they occur so 
that they can be mitigated to minimize any adverse impact. 

The improvement plan is currently being updated to reflect the results of 
the baseline assessment and to include a detailed work breakdown 
structure, process prioritization, and resource estimates. According to the 
Director, Acquisition and Program Management Office (APMO), the goal is 
to conduct a formal SEI appraisal to assess the capability level of some or 
all of the six processes by October 2006. 

Notwithstanding the recent steps to begin addressing our recommendation, 
much work remains to fully implement key acquisition management 
controls. Moreover, effectively implementing these controls takes 
considerable time. Therefore, it is important that these improvement 
efforts stay on track. Until these processes are effectively implemented, 
US- VISIT will be at risk of not delivering promised capabilities on time and 
within budget. 
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Determination and 
Disclosure of Whether 
Increments Produce 
Mission Value 
Commensurate with Costs 
and Risks Are Partially 
Complete 



In September 2003, we reported that the program had not assessed the 
costs and benefits of Increment 1, which is extremely important because 
the decision to invest in any capability should be based on reliable analyses 
of return on investment. Further, according to OMB guidance, individual 
increments of major systems are to be individually supported by analyses 
of benefits, cost, and risk. 23 Without reliable analyses, an organization 
cannot adequately know that a proposed investment is a prudent and 
justified use of limited resources. Accordingly, we recommended that DHS 
do the following: 



Determine whether proposed US- VISIT increments will produce mission 
value commensurate with cost and risks and disclose to the Congress 
planned actions. 



As we reported in September 2003 and again in February 2005, 24 the 
program office did not justify its planned investment in Increments 1 and 
2B, respectively, based on expected return on investment. Since then, the 
program has developed a cost-benefit analysis for Increment IB. 



OMB has issued guidance concerning the analysis needed to justify 
investments. 25 According to this guidance, such analyses should meet 
certain criteria to be considered reasonable. These criteria include, among 
other things, comparing alternatives on the basis of net present value and 
conducting uncertainty analyses of costs and benefits. DHS has also issued 
guidance on such economic analyses that is consistent with that of OMB. 26 



23 OMB, Planning, Budgeting, Acquisition and Management of Capital Assets, Circular 
A-ll, Part 7 (Washington, D.C.: June 21, 2005). 

24 GAO-05-202 and GAO-03-1083. 

25 OMB, Guidelines and Discount Rates for Benefits-Cost Analysis of Federal Programs, 
Circular A-94 (Washington, D.C.: Oct. 29, 1992). 

26 Department of Homeland Security, Capital Planning and Investment Control: 
Cost-Benefit Analysis Workbook (Washington, D.C.: May 2003). 
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The latest cost-benefit analysis for Increment IB (dated June 23, 2005) 
identifies potential costs and benefits for three exit solutions at air and sea 
POEs and provides a general rationale for the viability of the three 
alternatives described. This latest analysis meets four of eight OMB 
economic analysis criteria. However, it does not, for example, include a 
complete uncertainty analysis (i.e., both a sensitivity analysis and a Monte 
Carlo simulation 27 ) for the three exit alternatives evaluated. That is, the 
cost-benefit analysis does include a Monte Carlo simulation, but it does not 
include a sensitivity analysis for the three alternatives. An analysis of 
uncertainty is important because it provides decision makers with a 
perspective on the potential variability of the cost and benefit estimates 
should the facts, circumstances, and assumptions change. 

Table 1 summarizes our analysis of the extent to which US- VISIT'S June 23, 
2005, cost-benefit analysis for Increment IB satisfies eight OMB criteria. 



27 Uncertainty analyses generally include both a sensitivity analysis and a Monte Carlo 
simulation. A sensitivity analysis is a quantitative assessment of the effect that a change in 
an assumption — the numerical value of a single parameter (such as unit labor cost) — will 
have on net present value. A Monte Carlo simulation allows all of the model's parameters to 
vary simultaneously according to their associated probability distribution. The result is a set 
of estimated probabilities of achieving alternative outcomes (costs, benefits, and/or net 
benefits), given the uncertainty in the underlying parameters. 
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Table 1 : US-VISIT Satisfaction of OMB Economic Analysis Criteria 



Criterion 



Explanation 



Criterion met? 



GAO analysis 



1. 



The cost-benefit 
analysis clearly 
explained why 
the investment 
was needed. 



The analysis should clearly explain the reason 
why the investment is needed, that is, why the 
status quo is unacceptable. 



Yes 



The analysis identifies the need for the 
investment and identifies eight key 
business objectives of the Increment 1 B 
exit solution. 



At least two 
alternatives to 
the status quo 
were 

considered. 



At least two meaningful alternatives to the status 
quo should be examined to help ensure that the 
alternative chosen was not preselected. 



Yes 



The analysis considers three 
alternatives for the Increment 1 B exit 
solution: kiosk, mobile, and validator. 



The general 
rationale for the 
cost-benefit 
analysis, 
including each 
alternative, was 
discussed. 



The general rationale for the inclusion of each 
alternative considered should be discussed to 
enable reviewers of the analysis to gain an 
understanding of the context for the selection of 
one alternative over the others. 



Yes 



The assessment includes the rationale 
for the judgment that the three exit 
alternatives were viable options. 



The quality of 
the cost 
estimate for 
each alternative 
was reasonable. 



The quality of the cost estimate for each 
alternative should be complete and reasonable 
for a net present value to be accurate. 



No 



The cost estimates are not complete or 
reliably derived. (See later section of 
this report for detailed analysis.) 



The quality of The quality of the benefit estimate for each 
the benefits to alternative should be complete and reasonable 
be realized from for a net present value to be calculable and 
each alternative accurate. According to OMB Circular A-94, a 
was reasonable, year-by-year estimates should be reported to 

promote independent analysis and review of 

those estimates. 



No 



Year-by-year benefit estimates were not 
reported. 



Alternatives 
were compared 
on the basis of 
net present 
value. 



The net present value should be calculated 
because it consistently allows for the selection of 
the alternative with the greatest benefit net of 
cost. 



Yes 



Net present values were calculated for 
the three alternatives. However, the 
preferred alternative could not be 
selected on this basis, in part because 
the estimated net present value for all 
alternatives was negative. OMB 
guidance presumes that at least one will 
be positive, and that the selected 
alternative will have the greatest total 
benefit net of total cost. The alternative 
with the more favorable cost-benefit was 
identified on the basis of its lower labor 
intensity (resulting in lower operating 
and maintenance costs) and lower risk 
that personally identifiable information 
would be compromised. 
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(Continued From Previous Page) 



Criterion 



Explanation 



Criterion met? 



GAO analysis 



The proper 
discount rate for 
calculating each 
alternative's net 
present value 
should be used. 



OMB Circular A-94 provides specific guidance 
on the choice of discount rate for evaluating 
projects whose benefits and costs will be 
distributed over time. 



No 



The analysis does not explicitly state the 
numerical value of the discount rate 
used for computing the alternatives' net 
present values. 



A complete Estimates of costs and benefits are typically 
uncertainty uncertain because of imprecision in both 
analysis of cost underlying data and modeling assumptions, 
and benefit was Because such uncertainty is basic to virtually 
included. any cost-benefit analysis, its effects should be 

analyzed and reported. OMB guidance 
recommends both Monte Carlo simulation and 
sensitivity analysis as uncertainty analysis 
techniques. 



No 



Although the cost-benefit analysis did 
include Monte Carlo simulation results 
for the three exit alternatives, no 
sensitivity analysis was conducted for 
those alternatives. Instead, the 
cost-benefit analysis reports sensitivity 
analysis results for the five deployment 
scenarios. 



Source: GAO. 



a OMB's Circular A-94 is the general guidance for conducting cost-benefit analyses for the federal 
government. 



It is important that the program adhere to relevant guidance in developing 
its incremental cost-benefit analyses. If this is not done, the reliability of 
the analyses is diminished, and an adequate basis for prudent investment 
decision making does not exist. Moreover, if the mission value of a 
proposed investment is not commensurate with costs, it is vital that this 
information be fully disclosed to DHS and congressional decision makers. 
The underlying intent of our recommendation is that this information be 
available to inform such decisions. 



Definition of the 
Operational Context for 
US-VISIT Is in Progress 



In September 2003, we reported that key aspects of the larger homeland 
security environment in which US- VISIT would need to operate had not 
been defined. For example, we stated that certain policy and standards 
decisions had not been made (e.g., whether official travel documents will 
be required for all persons who enter and exit the country, including U.S. 
and Canadian citizens, and how many fingerprints are to be collected). In 
the absence of this operational context, program officials were making 
assumptions and decisions that, if they proved inconsistent with 
subsequent policy or standards decisions, would require US-VISIT rework. 
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To minimize the impact of these changes, we recommended that DHS do 
the following: 



Clarify the operational context in which US- VISIT is to operate. 



After about 27 months, defining this operational context remains a work in 
progress. According to the Chief Strategist, an immigration and border 
management strategic plan was drafted in March 2005 that shows how 
US-VISIT is aligned with DHS's organizational mission and defines an 
overall vision for immigration and border management. This official stated 
that this vision provides for an immigration and border management 
enterprise that unifies multiple internal departmental and other external 
stakeholders with common objectives, strategies, processes, and 
infrastructures. 

Since the plan was drafted, DHS has reported that other relevant initiatives 
have been undertaken, such as the Security and Prosperity Partnership of 
North America and the Secure Border Initiative. The Security and 
Prosperity Partnership is to, among other things, establish a common 
approach to securing the countries of North America — the United States, 
Canada, and Mexico — by, for example, implementing a border facilitation 
strategy to build capacity and improve the legitimate flow of people and 
cargo at our shared borders. The Secure Border Initiative is to implement a 
comprehensive approach to securing our borders and reducing illegal 
immigration. According to the Chief Strategist, while portions of the 
strategic plan are being incorporated into these initiatives, these initiatives 
and their relationship with US- VISIT are still being defined. We have yet to 
receive the US-VISIT strategic plan because, according to program officials, 
it had not yet been approved by DHS management. 

Until US- VISIT'S operational context is fully defined, DHS is increasing its 
risk of defining, establishing, and implementing a program that is 
duplicative of other programs and not interoperable with them. This in turn 
will require rework to address these areas. While this issue was significant 
27 months ago, when we made the recommendation, it is still more 
significant now. 



Page 24 



GAO-06-296 US-VISIT Recommendations 



Provision of Program Office 
Resources Is Partially 
Complete 



Ensure that human capital and financial resources are provided to 
establish a fully functional and effective program office. 



About 2 years later, US-VISIT had filled 102 of its 115 planned government 
positions and all of its planned 117 contractor positions. For the remaining 
13 government positions, 5 positions had been selected (pending 
completion of security clearances), and recruitment action was in process 
for filling the remaining 8 vacancies. According to the Office of 
Administration and Training Manager, funding is available to complete the 
hiring of all 115 government employees. 

Notwithstanding this progress, in February 2005, US- VISIT completed a 
workforce analysis and requested additional positions based on the results. 
According to program officials, a revised analysis was submitted in the 
summer of 2005, but the request has not yet been approved. Figure 3 shows 
the program office organization structure and functions and how many of 
the 115 positions needed have been filled. 



We reported in September 2003 that the program had not fully staffed its 
program office. Our prior experience with major acquisitions like US- VISIT 
shows that to be successful, they need, among other things, to have 
adequate resources. Accordingly, we recommended that DHS do the 
following: 
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Figure 3: Summary of Program Office Structure, Functions, and Filled and Vacant Positions 
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Securing necessary resources will be a continuing challenge and an 
essential ingredient to the program's ability to acquire, deploy, operate, and 
maintain system capabilities on time and within budget. 
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Definition of Program Office 
Roles and Responsibilities 
Has Been Completed 



We reported in September 2003 that the program had not defined specific 
roles and responsibilities for its staff Our prior experience and leading 
practices show that for major acquisitions like US-VISIT to be successful, 
program staff need, among other things, to understand what they are to do, 
how they relate to each other, and how they fit in their organization. 
Accordingly, we recommended that DHS do the following: 



Define program office positions, roles, and responsibilities. 



The program office has developed charters for its nine component offices 
that include roles and responsibilities for each. For example, the 
Acquisition and Program Management Office is responsible, among other 
things, for establishing acquisition and program management policies; 
coordinating development of configuration management plans and project 
schedules, including the integrated milestone schedule; and developing 
policies and procedures for guidance and oversight of systems 
development and implementation activities. The program has also defined 
a set of core competencies (knowledge, skills, and abilities) for each 
position. For example, it has defined critical competencies for program and 
management analysts that include, among others, flexibility, interpersonal 
skills, organizational awareness, oral communication, problem solving, and 
teamwork. 



These efforts to define position, roles, and responsibilities should help in 
managing the program effectively. 



Development and 
Implementation of a Human 
Capital Strategy Are 
Partially Complete 



As previously stated, we reported in September 2003 that US-VISIT had not 
fully staffed its program office or defined roles and responsibilities for its 
program staff. We observed that prior research and evaluations of 
organizations showed that effective human capital management can help 
agencies establish and maintain the workforce they need to accomplish 
their missions. Accordingly, we recommended that DHS do the following: 



Develop and implement a human capital strategy for the program office 
that provides for staffing positions with individuals who have the 
appropriate knowledge, skills, and abilities. 
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In February 2005, we reported that the program office, in conjunction with 
the Office of Personnel Management (OPM), developed a draft human 
capital plan that employed widely accepted human capital planning tools 
and principles. The draft plan included, for example, an action plan that 
identified activities, proposed completion dates, and the office (OPM or the 
program office) responsible for the action. We also reported that the 
program office had completed some of the activities, such as designating a 
liaison responsible for ensuring alignment between departmental and 
program human capital policies. 

Since then, the program office has finalized the human capital plan and 
completed more activities. For example, program officials told us that they 
have 

• analyzed the program office's workforce to determine diversity trends, 
retirement and attrition rates, and mission-critical and leadership 
competency gaps; 

• updated the program's core competency requirements to ensure 
alignment between the program's human capital and business needs; 

• developed an orientation program for new employees; and 

• administered competency assessments to incoming employees. 

Program officials also told us that they have plans to complete other 
activities, such as 

• developing a staffing forecast to inform succession planning; 

• analyzing workforce data to maintain strategic focus on preserving the 
skills, knowledge, and leadership abilities required for the US- VISIT 
program's success; and 

• developing organizational leadership competency models for the 
program's senior executive, managerial, and supervisory levels. 

In addition, the officials said that several activities in the plan have not 
been completed, such as assessing the extent of any current employees' 
competency gaps and developing a competency-based listing of training 
courses. These officials said that the reason these activities have not been 
completed is that they are related to the department's new human capital 
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initiative, MAX HR , which is to provide greater flexibility and accountability 
in the way employees are paid, developed, evaluated, afforded due process, 
and represented by labor organizations. MAX™ is to include the 
development of departmentwide competencies. Because of this, the 
officials told us that it could potentially impact the program's ongoing 
competency-related activities. As a result, these officials said that they are 
coordinating these activities closely with the department as it develops and 
implements this new initiative, which is currently being reviewed by the 
DHS Deputy Secretary for approval. 

Until US-VISIT fully implements a comprehensive human capital strategy, it 
will continue to risk not having staff with the right skills and abilities to 
successfully execute the program. 



Defining Performance 
Standards for US-VISIT 
Increments Is Partially 
Complete 



We reported in September 2003 that the operational performance of initial 
system increments was largely dependent on the performance of existing 
systems that were to be interfaced to create these increments. For 
example, we said that the performance of an increment will be constrained 
by the availability and downtime of the existing systems that it includes. 
Accordingly, we recommended that DHS do the following: 



Define performance standards for each increment that are measurable 
and reflect the limitations imposed by relying on existing systems. 



In February 2005 (17 months later), we reported that several technical 
performance standards for Increments 1 and 2B had been defined, but that 
it was not clear that these standards reflected the limitations imposed by 
the reliance on existing systems. Since then, for the Increment 2C Proof of 
Concept (Phase 1), the program office has defined certain other 
performance standards. For example, the functional requirements 
document for Increment 2C (Phase 1) defines several technical 
performance standards, including reliability, recoverability, and availability. 
For each, the document states that the performance standard is largely 
dependent on those of Increment 2B. More specifically, the document 
states that Phase 1 system availability is largely dependent upon the 
individual and collective availability of the current systems. The document 
also states that the Increment 2C components shall have an aggregated 
availability greater than or equal to 97.5 percent. However, the document 
does not contain sufficient information to determine whether these 
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performance standards actually reflect the limitations imposed by reliance 
on existing systems. 

To further develop performance standards, the program office has 
prepared a Performance Engineering Plan, dated March 31, 2005, that links 
US-VISIT performance engineering activities to its System Development 
Life Cycle. Further, the plan (1) provides a framework to be used to align 
its business, application, and infrastructure performance goals and 
measures; (2) describes an approach to translate business goals into 
operational measures, and then to quantitative metrics; and (3) identifies 
system performance measurement areas (effectiveness, efficiency, 
reliability, and availability). According to program officials, they intend to 
establish a group to develop action plans for implementing the engineering 
plan, but did not have a time frame for doing so. 

Without defining performance standards that reflect the limitations of the 
existing systems upon which US-VISIT relies, the program lacks the ability 
to identify and effectively address performance shortfalls. 



In September 2003, we reported that US-VISIT was a risky undertaking 
because of several factors inherent to the program, such as its large scope 
and complexity, as well as because of various program management 
weaknesses. We concluded that these risks, if not effectively managed, 
would likely cause program cost, schedule, and performance problems. 

Risk management is a continuous, forward-looking process that is intended 
either to prevent such problems from occurring or to minimize their impact 
if they occur by proactively identifying risks, implementing risk mitigation 
strategies, and measuring and disclosing progress in doing so. Because of 
the importance of effectively managing program risks, we recommended 
that DHS do the following: 



Develop and implement a risk management plan and ensure that all high 
risks and their status are reported regularly to the executive body. 



About 2 years later, the program office has developed and has begun 
implementing a risk management plan. The plan, which was approved in 
September 2005, includes, among other things, a process for identifying, 
analyzing, handling, and monitoring risk. It also defines the governance 



Development and 
Implementation of a Risk 
Management Plan Are 
Partially Complete 
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structure to be used in overseeing and managing the process. The program 
also maintains a risk database, which includes, among other things, a 
description of the risk, its priority (e.g., high, medium, or low), and its 
mitigation strategy. According to program officials, the database is 
currently available to program management and staff. 

The program has also begun implementing its risk management plan. For 
example, it has established a Risk Review Board, Risk Review Council, and 
Risk Owners to govern its risk activities. The roles and responsibilities are 
described below. 

• The Risk Review Board directs all risk governance within the program 
and provides the mechanism to escalate/transfer the consideration of 
risks to program governing boards and to organizations external to the 
program. 

• The Risk Review Council oversees and manages program-related risks 
that are significant, controversial, or cross-project or that may require 
escalation to the Risk Review Board. 

• Risk Owners analyze, handle, and monitor risks. 

However, full implementation of the risk management plan has yet to 
occur. As part of its CMMI process maturity baseline self-assessment 
(previously discussed), the program office found that the risk management 
process detailed in its plan was not being consistently applied across the 
program. In response, according to program officials, they have developed 
risk management training and began conducting training sessions in 
November 2005. These officials also stated that the Risk Review Board, 
where risks are reviewed with program executives, has been meeting 
monthly since September 2005. 

With respect to regular risk reports to program executives, the plan 
includes thresholds for escalating risks within the risk governance 
structure and to DHS governance entities. For example, risks are to be 
elevated to the Risk Review Board when the cost of the project exceeds 
more than 5 percent of the project baseline cost, the schedule slippage 
exceeds more than 5 percent of the baseline schedule, major areas of scope 
are affected, or quality reduction requires approval. However, program 
officials stated that these thresholds are not currently being applied. They 
further stated that although the plan allows for escalation of risks to 
officials outside the program office, doing so is at the discretion of the 
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Program Director; in addition, according to these officials, although high 
risks are not routinely escalated outside the program, selected high risks 
have been disclosed to the Assistant Secretary for Policy in weekly 
program status reports. As of December 5, 2005, the Program Director 
proposed submitting monthly reports of high-priority risks and issues 
through the Assistant Secretary for Policy to the Deputy Secretary. 

Until US- VISIT fully implements its risk management plan and process, it 
cannot be assured that all program risks are being identified and managed 
in order to effectively mitigate any negative impact on the program's ability 
to deliver promised capabilities on time and within budget. 



Development of Test Plans 
Is Partially Complete 



We reported in May 2004, and again in February 2005, that system testing 
was not based on well-defined test plans, and thus the quality of testing 
being performed was at risk. 28 The purpose of system testing is to identify 
and correct system defects (i.e., unmet system functional, performance, 
and interface requirements) and thereby obtain reasonable assurance that 
the system performs as specified before it is deployed and operationally 
used. To be effective, testing activities should be planned and implemented 
in a structured and disciplined fashion. Among other things, this includes 
developing effective test plans to guide the testing activities and ensuring 
that test plans are developed and approved before test execution. 
According to relevant systems development guidance, an effective test plan 
(1) specifies the test environment; (2) describes each test to be performed, 
including test controls, inputs, and expected outputs; (3) defines the test 
procedures to be followed in conducting the tests; and (4) provides 
traceability between the test cases and the requirements to be verified by 
the testing. Because these criteria were not being met, we recommended 
that DHS do the following: 



Develop and approve test plans before testing begins that (1) specify the 
test environment; (2) describe each test to be performed, including test 
controls, inputs, and expected outputs; (3) define the test procedures to 
be followed in conducting the tests; and (4) provide traceability between 
test cases and the requirements to be verified by the testing. 



2S GAO-05-202 and GAO-04-586. 
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About 19 months later, the quality of the system test plans, and thus system 
testing, is still problematic. To the program's credit, the test plans for the 
Increment 2C Proof of Concept (Phase 1), dated June 28, 2005, satisfied 
part of our recommendation. Specifically, the test plan for this increment 
was approved on June 30, 2005, and, according to program officials, testing 
began on July 5, 2005. Further, the test plan described, for example, the 
scope, complexity, and completeness of the test environment, and it 
described the tests to be performed, including a high-level description of 
controls, inputs, and outputs, and it identified test procedures to be 
performed. 

However, the test plan did not adequately trace between test cases and the 
requirements to be verified by testing. For example, 300 of the 438 
functional requirements, or about 70 percent of the requirements that we 
analyzed, did not have specific references to test cases. 

In addition, we identified traceability inconsistencies, including the 
following: 

• One requirement was mapped to over 50 test cases, but none of the 50 
cases referenced the requirement. 

• One requirement was mapped to a group of test cases in the traceability 
matrix, but several of the test cases to which the requirement was 
mapped did not reference the requirement, and several test cases 
referenced the requirement and were not included in the traceability 
matrix. 

• One requirement was mapped to all but one of the test cases within a 
particular group of test cases, but that test case did refer to the 
requirement. 

Time and resources were identified as the reasons that test plans have not 
been complete. Specifically, program officials stated that milestones do not 
permit existing testing/quality personnel the time required to adequately 
review testing documents. 29 According to these officials, even when the 
start of testing activities is delayed because, for example, requirements 



29 The Systems Assurance Manager stated that she has only two staff, including herself, for 
ensuring testing quality of the US- VISIT composite system. 
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definition or product development takes longer than anticipated, testing 
milestones are not extended. 

Without complete test plans, the program does not have adequate 
assurance that the system is being fully tested, and thus unnecessarily 
assumes the risk that system defects will not be detected and addressed 
before the system is deployed. This means that the system may not perform 
as intended when deployed, and defects will not be addressed until late in 
the systems development cycle, when they are more difficult and 
time-consuming to fix. As we previously reported, this has happened: 
postdeployment system interface problems surfaced for Increment 1, and 
manual work-arounds had to be implemented after the system was 
deployed. 



Assessment of the Impact of 
Increment 2B on Workforce 
Levels and Facilities Is 
Partially Complete 



We reported in May 2004 that the program had not assessed its workforce 
and facility needs for Increment 2B. Because of this, we questioned the 
validity of the program's workforce and facility assumptions used to 
develop its workforce and facility plans, noting that the program lacked a 
basis for determining whether its assumptions and thus its plans were 
adequate. Accordingly, we recommended that DHS do the following: 



Assess the full impact of Increment 2B on land POE workforce levels and 
facilities, including performing appropriate modeling exercises. 



Seven months later, the program office evaluated Increment 2B operational 
performance. The purpose of the evaluation was to determine the 
effectiveness of Increment 2B performance at the 50 busiest land POEs. To 
assist in the evaluation, the program office established a baseline for 
comparing the average Form 1-94 or Form I-94W 30 issuance processing 
times at 3 of the 50 POEs where processing times were to be evaluated. 31 
The program office then conducted two evaluations of the processing times 
at the 3 POEs following Increment 2B deployment. The first was in 
December 2004, after Increment 2B was deployed to these sites as a pilot, 
and the second was in February 2005, after Increment 2B was deployed to 



30 Form I-94W is used for foreign nationals from visa waiver countries. 
31 The sites were Douglas, Arizona; Port Huron, Michigan; and Laredo, Texas. 
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all 50 POEs. The evaluation results showed that the average processing 
times decreased for all 3 sites. Table 2 compares the results of the two 
evaluations and the baseline. 





Table 2: Reduction in Reported Processing Times for Increment 2B Pilot and Full Deployment 


Pilot site 


Baseline 
(October 2004) 


Pilot: 

Decrease in time from 
baseline 

(December 2004) 


Full deployment: 
Change in time 
from pilot 
(February 2005) 


Douglas, Arizona 


4 minutes, 16 seconds 


-47 seconds 


-17 seconds 


Laredo, Texas 


12 minutes, 10 seconds 


-9 minutes, 37 seconds 


-15 seconds 


Port Huron, Michigan 


1 1 minutes, 42 seconds 


-1 minutes, 51 seconds 


+7 seconds 



Source: GAO analysis of DHS data. 



According to program officials, these evaluations supported the workforce 
and facilities planning assumption that no additional staff were required to 
support deployment of Increment 2B, and that minimal modifications to 
interior workspace were required to accommodate biometric capture 
devices and printers and to install electrical circuits. These officials stated 
that modifications to existing officer training and interior space were the 
only changes needed. 

However, the scope of the evaluation was too limited to satisfy the 
evaluation's stated purpose or our recommendation for assessing the full 
impact of Increment 2B. Specifically, program officials stated that the 
evaluation focused on the time to process Form I-94s and not on 
operational effectiveness, including workforce impacts and traveler 
waiting time. Second, the 3 sites were selected, according to program 
officials, on the basis of a number of factors, including whether the sites 
already had sufficient staff to support the pilot. Selecting sites on the basis 
of this factor could affect the results and presupposes that not all POEs 
have the staff needed to support Increment 2B. Third, evaluation 
conditions were not always held constant. For example, fewer 
workstations were used to process travelers in establishing the baseline 
processing times at 2 of the POEs — Port Huron (9 versus 14) and Douglas 
(4 versus 6) — than were used during the pilot evaluations. 

Moreover, CBP officials from 1 POE, which was not an evaluation site, told 
us that US- VISIT has actually lengthened processing times. (San Ysidro 
processes the highest volume of travelers of all land POEs.) While these 
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officials did not provide specific data to support this statement, it 
nevertheless raises questions about the potential impact of Increment 2B 
on the 47 sites that were not evaluated. 

It is important that the impact of Increment 2B on workforce and facilities 
be fully assessed. Since we made our recommendation, Increment 2B 
deployment and operational facts and circumstances have materially 
changed, making the implementation of our recommendation using 
predeployment baseline data for the other 47 sites impractical. 
Nevertheless, other alternatives, such as surveying officials at these sites to 
better understand the increment's impact on workforce levels and 
facilities, have yet to be explored. Until they are, the program may not be 
able to accurately project resource needs or make required modifications 
to achieve its goals of minimizing US- VISIT'S impact on POE processing 
times. 



Implementation of 
Configuration Management 
Practices Is in Progress 



We reported in May 2004 that US-VISIT had not established effective 
configuration management practices. Configuration management 
establishes and maintains the integrity of system components and items 
(e.g., hardware, software, and documentation). A key ingredient is a change 
control board to evaluate and approve proposed configuration changes. 
Accordingly, we concluded that the program did not have adequate 
assurance that approved system changes were actually made, and that 
changes made to the component systems (for non-US- VISIT purposes) did 
not interfere with US- VISIT functionality. Accordingly, we recommended 
that DHS do the following: 



Implement effective configuration management practices, including 
establishing a US- VISIT change control board to manage and oversee 
system changes. 



After 19 months, US-VISIT has begun implementing configuration 
management practices. To its credit, the program recently issued a 
configuration management policy (September 2005) and prepared a draft 
configuration management plan (August 2005). The policy contains guiding 
principles, direction, and expectations for planning and performing 
configuration management, and includes activities, authorities, and 
responsibilities. The draft plan describes the configuration management 
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governance structure, including organizational entities and their 
responsibilities, the processes and procedures to be applied, and how 
controls are to be applied to products. The governance structure includes 
the Executive Configuration Control Board and the Configuration 
Management Impact Review Team. According to its charter, the 
configuration control board is responsible for determining the status of 
requested configuration changes and resolving any conflicts related to 
those changes for US-VISIT-managed systems (i.e., not for US- VISIT 
component systems managed by other DHS organizations). The Impact 
Review Team, which reports to the board, is responsible for reviewing 
requests for system changes and submitting a recommendation to the 
appropriate change review authority (i.e., either the US-VISIT control 
board or the control board in the DHS organization that manages the 
component system). According to program officials, for 
US-VISIT-managed systems, the review authority is the Executive 
Configuration Control Board. For other systems, such as TECS (which CBP 
manages), the US- VISIT review team may submit a recommendation to the 
appropriate control board (in this case, the CBP Control Board). 

The APMO director stated that the planned configuration management 
program is intended to complement rather than replace the configuration 
management programs for the legacy systems. That is, change requests 
approved by the US- VISIT Executive Configuration Control Board that 
require changes to a legacy system will be coordinated with the board 
having responsibility for that system. This means, however, that changes to 
component systems (e.g., IDENT, ADIS, and TECS) that are initiated and 
approved by another DHS organization, and that could affect US- VISIT 
performance, are not subject to US- VISIT configuration management 
processes and are not also being examined and approved by the US- VISIT 
control board. This lack of US- VISIT control was the impetus for our 
recommendation. 

Although US-VISIT has recently taken steps to begin addressing our 
recommendation, the program still does not adequately control changes to 
the component systems upon which US-VISIT performance depends. Until 
programwide configuration management practices are implemented, the 
program does not have an effective means for ensuring that approved 
system changes are actually made and that changes made to the 
component systems for non-US- VISIT purposes do not compromise 
US-VISIT functionality and performance. 
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Efforts to Ensure the 
Independence of the 
Verification and Validation 
Contractor Are Complete 



We reported in May 2004 that the program office's independent verification 
and validation (IV&V) contractor was not independent of the products and 
processes that it was verifying and validating. The purpose of IV&V is to 
provide management with objective insight into the program's processes 
and associated work products. Its use is a recognized best practice for 
large and complex system development and acquisition projects like 
US-VISIT. To be effective, the verification and validation function is to be 
performed by an entity that is independent of the processes and products 
that are being reviewed. Accordingly, we recommended that DHS do the 
following: 



Ensure the independence of the IV&V contractor. 



In July 2005, the program office issued a new contract for IV&V services. To 
ensure the contactor's independence, the program office (1) required that 
IV&V contract bidders be independent of the development and integration 
contractors; (2) reviewed each of the bidder's affiliations with the prime 
contract; (3) included provisions in the contract that prohibit the 
contractor from soliciting, proposing, or being awarded work (other than 
IV&V services) for the program; (4) required all contractor personnel to 
certify that they do not have any conflicts of interest; and (5) ensured that 
the contractor's management plan (Oct. 17, 2005) describes how the 
contractor will ensure technical, managerial, and financial independence. 



Such steps, if effectively enforced, should adequately ensure that 
verification and validation activities are performed in an objective manner 
and, thus, should provide valuable assistance to program managers and 
decision makers. 



Development of a Plan to 
Address Open 
Recommendations Is 
Partially Complete 



We reported in May 2004 that US-VISIT's overall progress on implementing 
our recommendations had been slow, and considerable work remained to 
fully address them. As we also noted, given that most of our 
recommendations focused on fundamental limitations in US-VISIT's ability 
to manage the program, it was important to implement the 
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recommendations quickly and completely. Accordingly, we recommended 
that DHS do the following: 



Develop a plan, including explicit tasks and milestones, for implementing 
all of our open recommendations and periodically report to the DHS 
Secretary and Under Secretary on progress in implementing this plan; 
and report this progress, including reasons for delays, in all future 
expenditure plans. 



About 19 months after our recommendation, the program assigned 
responsibility to specific individuals for preparing a plan, including specific 
actions and milestones, to address each recommendation. In addition, it 
developed a report that identifies the responsible person for each 
recommendation and summarizes progress made in implementing each. 
The program office provided this report for the first time to the DHS 
Deputy Secretary on October 3, 2005, and plans to forward subsequent 
reports every 6 months. 

However, the report's description of progress on 4 recommendations is 
inconsistent with our assessment, as discussed below: 

• First, the report states that the program completed a privacy impact 
assessment that is in full compliance with OMB guidance. As previously 
discussed, an assessment has been developed, but OMB guidance 
requires that these assessments for systems under development (such 
as Increment 2C) address privacy in the system's documentation. 
Increment 2C systems documentation does not address privacy and 
therefore is not fully compliant with OMB guidance. 

• Second, the report states that a human capital strategy has been 
completed. However, as previously discussed, several of the activities in 
the human capital plan have yet to be implemented. For example, the 
program has not developed a staffing forecast to inform succession 
planning. 

• Third, the report states that the impact of Increment 2B on land POE 
workforce levels and facilities has been fully assessed. However, as we 
previously stated, the scope of the evaluations was not sufficient to 
satisfy our recommendation. For example, program officials stated that 
the evaluation focused on the time to process Form I-94s and not on 
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operational effectiveness, including workforce impacts and traveler 
waiting time. Moreover, officials at the largest land POE told us that the 
effect of Increment 2B was the opposite of that reported in the pilot 
results. 

• Fourth, the report states that the program has partially completed 
implementing configuration management practices. However, as 
previously discussed, the program office has yet to implement practices 
or establish a configuration control board with authority over all 
changes affecting US-VISIT functionality and performance, including 
those made to component systems for non-US-VISIT purposes, which 
was the intent of our recommendation. 

In addition, the report does not specifically describe progress against 11 of 
our other recommendations, so that we could not determine whether the 
program's assessment is consistent with ours (described in this report). For 
example, we recommended that the program reassess plans for deploying 
an exit capability to ensure that the scope of the exit pilot provides for 
adequate evaluation of alternative solutions. The report states that the 
program office has completed exit testing and has forwarded the exit 
evaluation report to the Deputy Secretary for a decision. However, it does 
not state whether the program office had expanded the scope or time 
frames of the pilot. 

Fully understanding and disclosing progress against our recommendations 
are essential to building the capability needed to effectively manage the 
program, and to ensuring that key decision makers have the information 
needed to make well-informed choices among competing investment 
options. 



Establishment of Effective 
Cost-Estimating Practices Is 
in Progress 



We reported in February 2005 that US-VISIT had not followed effective 
practices to develop cost estimates for its system increments, and thus the 
reliability of its cost estimates was questionable. 32 Such cost-estimating 
practices are embedded in the 13 criteria in SEI's checklist for determining 
the reliability of cost estimates. 33 Of these 13 criteria, we reported in 



32 GAO-05-202. 

33 Carnegie Mellon University Software Engineering Institute, A Manager's Checklist for 
Validating Software Cost and Schedule Estimates, CMU/SEI-95-SR-004 (January 1995). 
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February 2005 that the program's cost estimate met 2, partially met 6, and 
did not meet 5. Accordingly, we recommended that DHS do the following: 



Follow effective practices for estimating the costs of future increments. 



The latest US-VISIT-related cost estimate is for Increment IB. This 
estimate is in the June 2005 cost-benefit analysis for Increment IB and 
establishes the costs associated with three exit solutions for air and sea 
POEs. As was the case for the estimate described in our February 2005 
report, this latest estimate also did not meet all 13 criteria, meeting 3 and 
partially meeting another 5. 34 For example, these estimates did not include 
a detailed work breakdown structure and omitted important cost elements, 
such as system testing. A work breakdown structure serves to organize and 
define the work to be performed, so that associated costs can be identified 
and estimated. Thus, it provides a reliable basis for ensuring that the 
estimates include all relevant costs. In addition, the uncertainties 
associated with the Increment IB cost estimate were not identified. An 
uncertainty analysis provides the basis for adjusting these estimates to 
reflect unknown facts and circumstances that could affect costs and 
identifies the risk associated with the cost estimate. Table 3 summarizes 
our analysis of the extent to which US-VISIT's Increment IB cost estimates 
satisfy SEI's 13 criteria. 



34 One criterion — when a dictated schedule is imposed, an estimate of the normal schedule is 
compared to the additional expenditures required to meet the dictated schedule — was not 
applicable because a schedule was not imposed. 
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Table 3: Satisfaction of SEI's 13 Cost-Estimating Criteria 



Criterion 



Explanation 



Criterion met? 3 



GAO analysis 



1 . The objectives of the program 
are stated in writing. 



The objectives of the program should 
be clearly and concisely stated for 
the cost estimator to use. 



Yes 



The objectives of the program were 
clearly stated. Specifically, the 
objectives are to provide a more 
complete traveler history and to 
capture travelers' biometric and 
biographic data. 



The life cycle to which the 
estimate applies is clearly 
defined. 



The life cycle should be clearly 
defined to ensure that the full cost of 
the program is captured — that is, all 
direct and indirect costs for planning, 
procurement, operations and 
maintenance, and disposal. 



Partially 



The life cycle was not clearly 
defined to ensure that the full cost 
of the program was included. For 
example, the analysis did not 
include evidence that software 
maintenance costs were included in 
the cost estimate. 



The task has been appropriately 
sized. 



An appropriate sizing metric should 
be used in the development of the 
estimate, such as the amount of 
software to be developed and the 
amount of software to be revised. 



No 



The program office provided no 
evidence to demonstrate that an 
appropriate sizing mechanism was 
used, and program officials stated 
that they had not collected these 
data. 



The estimated cost and 
schedule are consistent with 
demonstrated accomplishments 
on other projects. 



Estimates should be validated by 
being related back to demonstrated 
and documented performance on 
completed projects. 



Partially 



Officials stated that pilot data were 
used to develop the estimate. They 
stated they extrapolated pilot data 
to estimate costs for all Increment 
1B sites; however, they further 
stated that there were no previous 
projects with which to compare the 
results to see if they were 
consistent. 



A written summary of parameter 
values and their rationales 
accompanies the estimate. 



If a parametric equation was used to 
generate the estimate, the 
parameters that feed the equation 
should be provided, along with an 
explanation of why they were 
chosen. 



Partially 



High-level cost categories, such as 
labor, information technology, 
facilities, and other costs, were 
identified, but detailed parameters 
used to develop the estimate, such 
as number of software lines of 
code, which would be relevant to 
software maintenance costs, were 
not provided in the analysis. 



6. Assumptions have been 


Assumptions regarding issues such Yes 


General cost assumptions are 


identified and explained. 


as schedule, quantity, technology, 


identified and explained, as well as 




development processes, 


assumptions for workforce, 




manufacturing techniques, software 


information technology, training, 




language, etc., should be 


and facilities. 




understood and documented. 
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(Continued From Previous Page) 



Criterion 



Explanation 



Criterion met? a 



GAO analysis 



A structured process, such as a 
template or format, has been 
used to ensure that key factors 
have not been overlooked. 



A work breakdown structure or 
similar structure that organizes, 
defines, and graphically displays the 
individual work units to be performed 
should be used. The structure 
should be revised over time as more 
information becomes known about 
the work to be performed. 



Partially 



The analysis included four 
high-level cost categories (labor, 
facilities, operations and 
maintenance, and information 
technology), but it did not include a 
detailed work breakdown structure 
and omitted important cost 
elements, such as system testing. 



Uncertainties in parameter 
values have been identified and 
quantified. 



For all major cost drivers, an 
uncertainty analysis should be 
performed to recognize and reflect 
the risk associated with the cost 
estimate. 



Partially 



A risk analysis was performed, but 
this analysis did not identify 
detailed parameter values. 



9. If a dictated schedule has been 
imposed, an estimate of the 
normal schedule has been 
compared to the additional 
expenditures required to meet 
the dictated schedule. 



Managers should be informed of all 
potential cost savings associated 
with alternative schedules. 



N/A 



Program officials stated that the 
Increment 1B schedule was not 
dictated. 



1 0. If more than one cost model or 
estimating approach has been 
used, any differences in results 
have been analyzed and 
explained. 



The primary methodology or cost 
model results should be compared 
with any secondary methodology 
(e.g., cross checks) to ensure 
consistency. 



No 



No evidence of a secondary cost 
model was included in the analysis, 
and program officials stated that 
they did not use a second model. 



1 1 . Estimators independent of the 
performing organization 
concurred with the 
reasonableness of the 
parameter values and 
estimating methodology. 



The purpose of an independent 
estimate is to determine the 
reasonableness of the parameter 
values based on an unbiased 
perspective. This approach usually 
results in a more accurate estimate 
because it allows for better insight 
into program risks. 



No 



Program officials stated that the 
estimate was not independently 
reviewed. 



12. Estimates are current. 



Estimates are updated whenever 
changes to requirements affect cost 
or schedule, constraints, and 
resources, or when priorities change. 



Yes 



Estimates reflected current 
conditions. 



13. The results of the estimate have 
been integrated with project 
planning and tracking. 



Plans are reviewed and updated 
whenever estimates change, and 
estimates used for project planning 
are also used as baselines for 
project tracking. 



No 



Program officials stated that the 
results of the estimate have not 
been incorporated with project 
planning. 



Source: GAO. 



a We assessed each of the criteria as satisfied (US-VISIT provided substantiating evidence for the 
criterion), partially satisfied (US-VISIT provided partial evidence, including testimonial evidence, for the 
criterion), or not satisfied (no evidence was found for the criterion). 



Program officials stated that they recognize the importance of developing 
reliable cost estimates and have initiated actions to more reliably estimate 
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the costs of future increments. For example, as part of its process 
improvement program, the program has chartered a cost-analysis process 
action team, which is to develop, document, and implement a cost-analysis 
policy, process, and plan for the program. Program officials also stated that 
they have hired additional contracting staff with cost-estimating 
experience. 

Strengthening the program's cost-estimating capability is extremely 
important. The absence of reliable cost estimates, among other things, 
prevents the development of reliable economic justification for program 
decisions and impedes effective performance measurement. 



Reassessment of Plans for 
Deploying the Exit 
Capability Is Partially 
Complete 



In February 2005, we reported that US- VISIT had not adequately planned 
for evaluating the Increment IB exit alternative because its exit pilot 
evaluation's scope and timeline were compressed. Accordingly, we 
recommended that DHS do the following: 



Reassess plans for deploying an exit capability to ensure that the scope 
of the exit pilot provides for adequate evaluation of alternative solutions 
and better ensures that the exit solution selected is in the best interest of 
the program. 



Over the last 10 months, the program office has taken actions to expand the 
scope and time frames of the pilot. For example, it extended the pilot from 
5 to 11 POEs — 9 airports and 2 seaports. 35 It also extended the time frame 
for data collection and evaluation to April 2005, which is about 7 months 
beyond the date for which all exit pilot evaluation tasks were to be 
completed. Further, according to program officials, they achieved the 
target sample sizes necessary to have a 95 percent confidence level. 



35 The initial plan was to expand the pilot to 15 sites, but 4 of the sites were not fully 
operational in time to be evaluated. According to the Pilot Evaluation Report, this was 
largely due to the lengthy security clearance process for workstation attendants, who assist 
travelers in using one of the exit devices. 
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Notwithstanding the expanded scope of the pilot, questions remain about 
whether the exit alternatives have been evaluated sufficiently to permit 
selection of the best exit solution for national deployment. For example, 
each of the three exit alternatives was evaluated against three criteria, 
including compliance with the US- VISIT exit process (i.e., foreign travelers 
providing information as they exit the United States). 36 However, across the 
three alternatives, the average compliance with this process was only 24 
percent, which raises questions as to the effectiveness of the three 
alternatives. 37 The evaluation report cites several reasons for the low 
compliance rate, including that compliance during the pilot was voluntary. 
The report further concludes that national deployment of the exit solution 
will not have the desired compliance rate unless the exit process 
incorporates an enforcement mechanism, such as not allowing persons to 
reenter the United States if they do not comply with the exit process. 
Although an enforcement mechanism might indeed improve compliance, 
program officials stated that no formal evaluation has been conducted of 
enforcement mechanisms or their effect on compliance. The program 
director stated that he agrees that additional evaluation is needed to assess 
the impact of implementing potential enforcement mechanisms and plans 
to do so. 

Until the program office adequately evaluates the exit alternatives and 
knows whether the alternative to be selected will be effective, the program 
office will not be in a position to select the exit solution that is in the best 
interest of the program. This is very important because without an effective 
exit capability, the benefits and the mission value of US- VISIT are greatly 
diminished. 



We reported in February 2005 that the overall capacity of the system was 
not being effectively managed. At that time, US-VISIT, which comprises 
several legacy systems, was relying on the capacity management activities 
of these systems. It was not focused on the capacity requirements and 
performance of the collective systems that make up US-VISIT. This 
approach increases the risk that the system may not be properly designed 
and configured for efficient performance, and that it has insufficient 



36 The other two evaluation criteria were conduciveness to travel and cost. 

37 Compliance rate for kiosk was 23 percent; for the mobile device, 36 percent; and for the 
validator, 26 percent. 



Development and 
Implementation of Capacity 
Management Processes Are 
in Progress 
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processing and storage capacity for current, future, and unpredictable 
workload requirements. Accordingly, we recommended that DHS do the 
following: 



Develop and implement processes for managing the capacity of the 
US-VISIT system. 



According to program officials, they have initiated efforts to develop a 
capacity management process, including a high-level description of the 
necessary steps, such as identifying tools needed to implement the process. 
However, a plan, including specific tasks and milestones for developing and 
implementing capacity management processes, has not yet been 
developed. 

Until the program office develops a programwide capacity management 
program, it increases the risk that US- VISIT may not be able to adequately 
support program mission needs. 



Identification of ACE and 
US-VISIT Relationships and 
Dependencies Is in Progress 



We reported in February 2005 that the program office recognized that 
US- VISIT and the Automated Commercial Environment (ACE) 38 have 
related missions and operational environments. In addition, US-VISIT and 
ACE could potentially develop, deploy, and use common information 
technology infrastructures and services. We also reported that managing 
this relationship has not been a priority. Accordingly, we recommended 
that DHS do the following: 



Make understanding the relationships and dependencies between the 
US- VISIT and ACE programs a priority matter, and report periodically to 
the Under Secretary on progress in doing so. 



US- VISIT and ACE managers met in February 2004, to identify potential 
areas for collaboration between the two programs and to clarify how the 
programs could best support the DHS mission and provide officers with the 



38 ACE is a new trade processing system planned to support the movement of legitimate 
imports and exports and strengthen border security. 
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information and tools they need. According to program officials, they have 
established a US-VISIT/ACE integrated project team to, among other 
things, ensure that the two programs are programmatically and technically 
aligned. The team has discussed potential areas of focus and agreed to 
three areas: RF technology, program control, and data governance. 
However, it does not have an approved charter, and it has not developed 
explicit plans or milestone dates for identifying the dependencies and 
relationships between the two programs. Program officials stated that the 
team has met three times and plans to meet on a quarterly basis going 
forward. 

It is important that the relationships and dependencies between these two 
programs be managed effectively. The longer it takes for the programs to 
understand and exploit their relationships, the more rework will be needed 
at a later date to do so. 



Conclusions Over the last 3 years, we have made recommendations aimed at correcting 

fundamental limitations in US- VISIT'S program management ability and 
thereby better ensuring the delivery of mission capability and value on time 
and commensurate with costs. While progress on the implementation of 
the recommendations is mixed, progress in critical areas has been slow. As 
with any program, introducing and institutionalizing the program 
management and accountability discipline at which our recommendations 
are aimed require investing time and resources while continuing to meet 
other program demands. In making such investment choices, it is important 
to remember that institutionalizing such program discipline in the near 
term will produce long-term payback in a program's ability to meet these 
other demands. Accordingly, the longer that US-VISIT takes to implement 
our recommendations, the greater the risk that the program will not meet 
its stated goals and commitments. 

Our open recommendations are all aimed at strengthening US-VISIT 
program management and improving DHS's ability to make informed 
US-VISIT investment decisions. With the exception of one, these 
recommendations are still relevant and applicable. Since we made our 
recommendation, facts and circumstances surrounding Increment 2B 
deployment and operational status have materially changed, making the 
collection of Increment 2B predeployment impractical. Nevertheless, the 
need remains to better understand the impact of US- VISIT entry 
capabilities on all land POEs. Until this understanding exists, the 



Page 47 



GAO-06-296 US-VISIT Recommendations 



department will be challenged in its ability to accurately estimate and 
provide facilities and staff resource needs. 



To recognize both the need to fully assess the impact of US-VISIT entry 
capabilities on staffing levels and facilities at land POEs, as well as the 
current operational status of Increment 2B, we are closing our existing 
recommendation related to assessing the impact of Increment 2B. We 
recommend that the DHS Secretary direct the US- VISIT Program Director 
to explore alternative means of obtaining an understanding of the full 
impact of US-VISIT at all land POEs, including its impact on workforce 
levels and facilities; these alternatives should include surveying the sites 
that were not part of the previous assessment. 



In its written comments on a draft of this report, signed by the Director, 
Departmental GAO/OIG Liaison Office, and reprinted in appendix II, DHS 
stated that it agreed with many areas of the report and that our 
recommendations had made US-VISIT a stronger program. Further, the 
department stated that while it disagreed with certain areas of the report, it 
nevertheless concurred with the need to implement our open 
recommendations with all due speed and diligence. 

DHS commented specifically on 11 of the 18 recommendations discussed in 
the report. The recommendations, the department's comments, and our 
responses follow: 

1. Recommendation: Develop and begin implementing a system security 
plan, and perform a privacy impact assessment and use the results of 
the analysis in near-term and subsequent system acquisition decision 
making. 

DHS stated that this recommendation has been fully implemented. In 
support, it said that it has completed a US- VISIT security plan that is 
consistent with National Institute of Standards and Technology (NIST) 
guidance, and that it provided the plan to us in September 2004. It also 
stated that the security risk assessment aspect of this recommendation was 
established in February 2005, 20 months after we made the 
recommendation, and thus the age of the recommendation should be 
shown as 10 months rather that the 30 months cited in the report. 



Recommendation for 
Executive Action 



Agency Comments and 
Our Evaluation 
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The department also commented that there is no US- VISIT system, but 
rather a US- VISIT program with capabilities delivered by existing 
interconnected systems. According to the department, these component 
systems have been certified and accredited, consistent with NIST guidance, 
and as part of their certification and accreditation, security plans and risk 
assessments, as well as risk mitigation strategies, have been developed for 
each system. The department stated that it provided us with these 
system-level risk assessments, as well as system-specific action plans and 
milestones for implementing the mitigation strategies. In addition, the 
department noted that it completed a programwide risk assessment in 
December 2005 that specifically addresses information security issues that 
might not be captured in the system-specific documentation used to certify 
and accredit each system. In light of its system-specific certification and 
accreditation efforts, existing system-level risk assessments, and the 
program-level risk management process (see response 4 for discussion of 
the risk management process), DHS commented that it is inaccurate to 
state that US- VISIT officials are not in a position to know program risks, 
and the recommendation should be closed. 

While we agree that we received a copy of the US-VISIT security plan, 
dated September 2004, we do not agree that the plan satisfied all relevant 
federal guidance and that DHS has fully implemented our recommendation. 
In particular, it has not provided us with evidence that a programwide risk 
assessment has been done and that a security plan reflective of such an 
assessment exists. According to relevant guidance, 39 a security plan should 
describe, among other things, the methodology that is to be used to identify 
system threats and vulnerabilities and to assess risks, and it should include 
the date the risk assessment was completed because the assessment is a 
necessary driver of the security controls described in the plan. As we 
reported in February 2005 and state in this report, the US-VISIT security 
plan did not include this information; further, although DHS stated in its 
comments that it completed this risk assessment in December 2005, this 
statement is contradicted by a statement elsewhere in its comments that it 
is still in the process of doing the assessment. In addition to this 
contradiction, DHS's comments did not include any evidence to 



39 OMB, Security of Federal Automated Information Resources, Circular A-130, Revised 
(Transmittal Memorandum No. 4), Appendix III (Washington, D.C.: Nov. 28, 2000); and 
National Institute of Standards and Technology, Guide for Developing Security Plans for 
Information Technology Systems, Special Publication 800-18 (December 1998). 
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demonstrate that it has developed a complete risk assessment, such as a 
copy of the assessment. 

With regard to the age of the recommendation, we do not agree with DHS's 
position that we established a new finding regarding the lack of a 
programwide risk assessment in our February 2005 report. Rather, as part 
of our analysis of actions to implement our prior recommendation to 
develop a security plan, which is to include information about the related 
security risk assessment, we observed that the plan did not indicate a date 
for completing a risk assessment in accordance with federal guidelines. 
Therefore, our position that about 30 months had passed from the time of 
our initial recommendation (June 2003) is accurate. 

With regard to the individual system-level risk assessments, we agree that 
we have received them. However, we do not agree that we have received 
the action plans and milestones cited in the comments. Regardless, we do 
not believe that system-level assessments are a sufficient substitute for a 
programwide assessment. Accordingly, our recommendation focused on 
the need for an integrated US- VISIT system risk assessment as part of 
security planning. While the system-level plans and risk assessments are 
relevant and useful, they neither individually nor collectively address the 
threats and vulnerabilities imposed as a result of these systems' 
integration. By stating in its comments its commitment to having a 
programwide risk assessment that identifies and proposes mitigations for 
security risks that arise as a result of the interface and integration of the 
legacy systems, DHS is agreeing with our position. Moreover, without 
evidence that the program has completely assessed its risks, we continue 
to find no basis for how program officials would know the full range and 
degree of US- VISIT security risks. Our position in this regard has been 
reinforced by a recent DHS Inspector General report that identified a 
number of US- VISIT security risks. 40 

To further support its position that this recommendation has been fully 
implemented, DHS also commented that it has completed numerous 
privacy impact assessments and continues to update them to reflect system 
changes. In particular, it said that it updated the privacy impact assessment 
in December 2005 to reflect all increments and that it considers the 



40 Department of Homeland Security, US-VISIT System Security Management Needs 
Strengthening (Redacted), Office of Inspector General, OIG-06-16 (Washington, D.C.: 
December 2005). 
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assessment to be part of US-VISIT system documentation. It further 
commented that we appear to be unaware of privacy staff activities to 
review system documents and perform privacy risk assessments 
throughout the system life cycle. Nevertheless, the department 
acknowledged that its privacy work was not always noted within US-VISIT 
system documentation. Accordingly, DHS stated that it plans to 
appropriately reference all privacy requirements and privacy risk 
assessments in the program's system documentation in the future. 

We agree that US- VISIT has developed and updated its privacy impact 
assessment and would note that our report states this fact. We do not agree, 
however, with the comment that we are not aware that the privacy staff 
review system documents and perform privacy risk assessments. In fact, it 
is because we were aware of these facts that we were careful to ensure that 
they were reflected in our report. The point that we are making is that 
privacy is not addressed in all relevant systems documentation, which DHS 
acknowledged in its comments. With regard to this point of agreement, we 
support the department's stated plans to reference all privacy requirements 
and any privacy risk assessments in all relevant system documentation in 
the future. 

2. Recommendation: Develop and implement a plan for satisfying key 
acquisition management controls, including acquisition planning, 
solicitation, requirements management, program management, 
contract tracking and oversight, evaluation, and transition to support, 
and implement the controls in accordance with SEI guidance. 

DHS commented that the report should reflect that US-VISIT had initially 
adopted Carnegie Mellon University's Software Engineering Institute (SEI) 
Software Acquisition Capability Maturity Model® to guide its 
software-related process improvement efforts and that, in December 2004, 
it transitioned to SEI's Capability Maturity Model-Integration (CMMI®). As 
a result, it said that the program's process improvement strategy and plans, 
process development, and process appraisals are now aligned to the most 
applicable CMMI process areas. 

We agree that US-VISIT has transitioned to CMMI. We state in our report 
that US- VISIT has done so and that the key process areas it is addressing in 
its process improvement strategy and plan are consistent with those cited 
in our recommendation. We do not believe that this transition materially 
affects our recommendation, however, because even though the names of 
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the key processes in these two models may in some cases differ, the 
processes and respective practices are fundamentally consistent. 

3. Recommendation: Clarify the operational context in which US-VISIT is 
to operate. 

Consistent with our report, DHS commented that the operational context 
in which US-VISIT operates is in progress, meaning that it has yet to be 
fully established. For example, it said that the mission of DHS, and 
therefore the scope of US- VISIT activities to meet the mission, is 
continually expanding. Further, it acknowledged that more certainty in the 
operational context is desirable. In mitigation of the risks associated with 
not having a more stable operational context, DHS made several 
statements. For example, it said that the principal role of US-VISIT is to 
integrate information and immigration and border management systems 
across DHS and the State Department, and to facilitate agencies working 
toward a common environment that will eliminate redundancies. It also 
said that elements of its draft immigration and border management 
strategic plan are being used in current US- VISIT operations. In addition, 
the department said that mechanisms to mitigate the risks that we cited 
have been developed and are being implemented. 

We support DHS's acknowledgment of the importance of having a 
well-defined operational context within which to define and implement 
US-VISIT and related border security programs. However, we do not 
believe that DHS's comments provided any evidence showing that 
sufficient steps and activities to mitigate the associated risks have been 
taken or are planned. 

4. Recommendation: Determine whether proposed US-VISIT increments 
will produce mission value commensurate with cost and risks and 
disclose to the Congress planned actions. 

DHS commented that its cost-benefit analysis (CBA) for Increment IB 
conforms to relevant federal guidance, and noted that our expectations as 
to the scope and level of detail of analysis that should be included in the 
CBA document are inconsistent with its understanding of OMB Circular 
A-94 
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and DHS's CBA workbook, 41 which were used to guide the development of 
the CBA analysis. As an example, the department took exception with our 
statement that year-by-year benefit estimates were not reported by noting 
that the net present value was based on an estimate of annual benefits and 
costs, and that net present value could not be estimated without a 
year-by-year benefit analysis. 

The department further commented that a comprehensive uncertainty 
analysis was conducted because it completed a risk analysis, which is more 
comprehensive, rigorous, and appropriate than conducting a sensitivity 
analysis. In this regard, it added that the results of the risk analysis 
provided an indication of Increment lB's worthiness in light of existing 
uncertainty, rather than information on a specific CBA variable or another. 
The department further noted that it had provided some of these 
supporting analyses to us. 

DHS also stated that any investment that has a 5-year life cycle and is 
considered interim in nature will face considerable challenge in providing 
economic benefits commensurate with cost. 

We do not agree that the CBA fully conforms to relevant federal guidance. 
As our report states, for example, the analysis does not explicitly state the 
numerical value of the discount rate used for calculating each alternative's 
net present value, and hence does not conform to OMB guidance. In 
addition, the cost estimates used in the analysis were not complete and 
reliably derived. In deriving the estimate, for example, the department did 
not clearly define the project's life cycle to ensure that key factors were not 
overlooked and that the full cost of the program was included. (See 
response 10 below for more information on this point.) Last, while we 
agree that a year-by-year benefit analysis is a necessary component of a net 
present value determination, OMB nevertheless requires that the 
year-by-year benefit estimates be reported in the analysis to promote 
independent review of the estimates. 

Also, we do not agree that DHS performed a complete uncertainty analysis. 
According to OMB and DHS guidance, a complete uncertainty analysis 



41 OMB, Guidelines and Discount Rates for Benefits-Cost Analysis of Federal Programs, 
Circular A-94 (Washington, D.C.: Oct. 29, 1992); and Department of Homeland Security, 
Capital Planning and Investment Control: Cost-Benefit Analysis Workbook (Washington, 
D.C.: May 2003). 
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should include both a risk analysis and a sensitivity analysis. However, the 
latter was not done. Thus, our point is not, as DHS comments suggest, that 
US-VISIT should have performed a sensitivity analysis instead of a risk 
analysis, but rather, that both types of analyses are necessary to completely 
examine investment uncertainty. 

5. Recommendation: Develop and implement a risk management plan and 
ensure that all high risks and their status are reported regularly to the 
executive body. 

DHS commented that US- VISIT began the development and 
implementation of its risk management plan in 2004 immediately after we 
made our recommendation. It further commented that, as part of a CMMI 
maturity internal appraisal that it completed in July 2005, it found that the 
risk management process had not been consistently applied across the 
program. To address this, the department cited actions that it has taken to 
fully implement risk management, such as approving the risk management 
plan in September 2005; defining a risk governance structure; establishing 
and maintaining a risk database; and developing risk management training 
and providing this training to program personnel and contractors beginning 
in November 2005. 

We support the recent actions that the program cited as having been taken 
to strengthen risk management. However, the actions cited do not 
demonstrate that the risk management process is being consistently 
applied. Until US-VISIT fully implements its risk management plan and 
process, it cannot be assured that all program risks are being identified and 
managed in order to effectively mitigate any negative impact on the 
program's ability to deliver promised capabilities on time and within 
budget. 

6. Recommendation: Develop and approve test plans before testing 
begins that (1) specify the test environment; (2) describe each test to be 
performed, including test controls, inputs, and expected outputs; 

(3) define the test procedures to be followed in conducting the tests; 
and (4) provide traceability between test cases and the requirements to 
be verified by the testing. 

DHS stated that our report does not accurately reflect the status of the 
Increment 2C Phase 1 testing. In particular, it said that the issues 
associated with the traceability of requirements to test cases were minor 
and that the extent of the discrepancies is far less than what our report 
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presents. It further stated that the discrepancies in our report are based on 
old traceability documentation and do not reflect revised documentation 
provided to us on November 9, 2005. 

We agree that DHS provided us with revised traceability matrixes after we 
had shared with them our analysis of the test plans and traceability 
matrixes, dated June 28, 2005, and June 27, 2005, respectively. However, the 
revised documentation referenced in DHS's comments was provided in 
November 2005, about 4 months after testing began. This means that the 
test plans and traceability matrixes available at the time of testing — which 
are what we reviewed because they governed the scope and nature of 
actual testing performed — did not adequately trace between test cases and 
the requirements to be verified. Specifically, 300 of the 438 Increment 2C 
requirements, or about 70 percent, did not have specific references to test 
cases. 

7. Recommendation: Implement effective configuration management 
practices, including establishing a US-VISIT change control board to 
manage and oversee system changes. 

DHS commented that a US- VISIT representative attends all configuration 
control board meetings for all applicable legacy component systems, and 
that any proposed change request from a legacy component control board 
that could affect US- VISIT functionality is brought to the attention of the 
US-VISIT Executive Configuration Control Board for consideration. 

We do not question these statements. However, we do not believe that they 
demonstrate that US- VISIT has adequate control over system changes that 
could affect the program. That is, they do not ensure that changes to the 
component systems that are initiated and approved by another DHS 
organization and that could affect US- VISIT performance are subject to 
US-VISIT configuration management and approval processes. US- VISIT 
could establish explicit and enforceable control over changes to the legacy 
systems through such mechanisms as defined and enforced memorandums 
of understanding among the affected DHS organizations. It was the lack of 
such control that prompted our recommendation. 

8. Recommendation: Assess the full impact of Increment 2B on land POE 
workforce levels and facilities, including performing appropriate 
modeling exercises. 
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The department stated that, given the imperative to meet the legislatively 
mandated time frames, the scope of Increment 2B was limited to only one 
part of POE operations — incorporating the collection of a biometric into 
the previously manual Form 1-94 issuance process. It also stated that wait 
times are affected by various factors, including traffic volume, staffing 
levels, and availability of officers. Therefore, DHS focused the Increment 
2B evaluation on just the change to this process. 

The department further commented that given the events since the 
evaluation — namely, Increment 2B full operations — it is not practical to 
collect and model baseline data for the 47 sites that were not part of the 
initial evaluation. 

Regarding the 3 pilot sites included in the assessment, the department 
stated that the sites were selected based on criteria developed from input 
from US-VISIT, as well as CBP operational constraints. The department 
further commented that the 3 sites provided a reasonable mix of travelers 
and they did not have other constraints that directly impacted the 
collection of performance data specific to Form 1-94 issuance. DHS also 
stated that the 1-94 processing times vary by POE, and therefore they are 
not easily generalized from one port to another. Further, the department 
commented that the number of workstations and officers available to 
operate those workstations to process applicants for a Form 1-94 do not 
impact the time it takes to issue a Form 1-94. 

We agree that the scope of the Increment 2B evaluation was limited to the 
1-94 issuance process, and that it did not address the increment's impact on 
the POEs' ability to meet other performance parameters. Our point is that 
the limited nature of the evaluation does not satisfy either the intent of our 
recommendation or DHS's own stated purpose for the evaluation, which 
was to determine the effectiveness of Increment 2B performance at the 50 
busiest land POEs. We also agree that the 1-94 processing times vary by 
POE and cannot be easily generalized. It is for this reason, among others, 
that we questioned whether the 3 sites selected for the assessment were 
sufficiently representative to satisfy both our recommendation and the 
evaluation's stated purpose. 

In addition, while we also agree that collecting pre-Increment 2B baseline 
data is not practical at this time, the fact remains that the operational 
impact of Increment 2B on workforce levels and facilities has not been 
adequately assessed, as evidenced by officials at 1 large POE telling us that 
processing times have increased and DHS's recognition that each POE is 
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somewhat different. In light of these new facts and circumstances, we are 
closing our existing recommendation and making a new recommendation 
to recognize the need for DHS to explore alternative means to assess the 
impact of US-VISIT entry capabilities at land POEs. This new 
recommendation will be shown as an open recommendation, and the 
original recommendation will be closed. 

9. Recommendation: Develop a plan, including explicit tasks and 
milestones, for implementing all of our open recommendations and 
periodically report to the DHS Secretary and Under Secretary on 
progress in implementing this plan; and report this progress, including 
reasons for delays, in all future expenditure plans. 

DHS stated that it is untrue that 19 months had elapsed from the time we 
made this recommendation to the time that it assigned responsibilities to 
program officials for addressing each of our recommendations. In support, 
it commented that it issued its first plan to address our recommendations 
on August 18, 2003, and subsequent reports have been issued periodically 
that update progress in doing so. 

We agree that DHS has assigned responsibilities to specific individuals for 
addressing each recommendation. However, we have yet to be provided 
any evidence to support its statement that it issued the first report 
addressing our recommendations on August 18, 2003. Similarly, we have 
not received evidence showing that it has prepared a plan, including 
specific actions and milestones, for implementing all of our open 
recommendations, which is a focus of this recommendation. We would also 
observe that we made this recommendation in May 2004, and at that time 
the department stated that it agreed with the recommendation but did not 
indicate that it had taken any steps to address it, such as commenting that a 
report was issued on August 18, 2003. 

10. Recommendation: Follow effective practices for estimating the costs 
of future increments. 

DHS either tacitly or explicitly agreed with our findings relative to its 
satisfaction of 8 of the 13 cost-estimating criteria presented in table 4 (now 
table 3) of our draft report. For example, it agreed that it did not clearly 
define the life cycle to which the cost estimate applies. It also agreed that it 
did not include a work breakdown structure, noting that it used the 
available project implementation schedule as a proxy for the activities 
related to the deployment of the exit alternatives. 
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Regarding our five findings concerning its satisfaction of cost-estimating 
with which DHS disagreed, the department's primary area of disagreement 
was with the intended purpose of the Increment IB CBA that used the cost 
estimate, which it said in its comments was to inform decision makers 
about the relative worthiness of each of the three exit alternatives 
considered for deployment. Hence, DHS stated that the purpose of the CBA 
was to analyze only the costs associated with deploying an operational 
solution, not to analyze the costs and benefits of both developing and 
deploying alternative solutions. DHS further stated that the CBA thus 
includes only those costs to be incurred in deploying a selected alternative, 
and it does not include costs already incurred in developing system 
alternatives (i.e., sunk costs). It further commented that DHS guidance 
states that sunk costs are not relevant to the current investment analysis 
because "only current decisions can affect the future consequences of 
investment alternatives." 

DHS also disagreed that the cost estimate in the CBA should have included 
nonrecurring development costs, and commented that it did appropriately 
size the task described in the cost estimates for each alternative exit 
solution, noting that sizing metrics related to software development were 
not relevant to deployment of the alternatives because development 
activities had already occurred and therefore are sunk costs. The 
department added that those sizing metrics that are relevant to the cost 
estimate are discussed in the CBA, as are the cost estimating parameters 
(i.e., those associated with deployment and not those associated with 
development and testing). 

In addition, DHS disagreed that DHS's cost estimate excluded important 
cost categories, such as system testing, and stated that the estimate 
addresses labor, facilities, operations and maintenance, information 
technology, travel, and training costs. Once again, DHS emphasized that 
since the focus of the CBA was on operational deployment and not system 
design and development, system testing costs were not included because 
they were not considered relevant. DHS also reiterated its early point that 
the uncertainty analysis that it conducted was comprehensive. 

We agree that actual sunk costs should not be included in a CBA cost 
estimate. However, we disagree that the cost categories that DHS cited as 
not relevant are only costs that are associated with predeployment 
activities. Testing, for example, is an activity that is normally performed 
before, during, and following deployment, and thus the associated costs 
would be relevant to the stated purpose of the Increment IB CBA. 
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However, a testing cost category was missing from the CBA cost estimate, 
as was a cost category for software maintenance. 

Regarding DHS's statement that it conducted a complete uncertainty 
analysis, we reiterate our previous point that a complete uncertainty 
analysis should include both a risk analysis and a sensitivity analysis, and 
the CBA did not include the latter. 

11. Recommendation: Reassess plans for deploying an exit capability to 
ensure that the scope of the exit pilot provides for adequate evaluation 
of alternative solutions and better ensures that the exit solution 
selected is in the best interest of the program. 

Concerning the questions we raised about the adequacy of the exit pilots in 
light of the 24 percent compliance rate, DHS commented that we failed to 
consider the compliance rate of the previous exit pilot program, the 
National Security Entry Exit Registration System (NSEERS), which, 
according to DHS, had a 75 percent compliance rate. DHS added that 
NSEERS achieved this compliance rate with a very limited number of exit 
locations, and therefore, any of the three US- VISIT exit alternatives would 
have at least a 75 percent compliance rate once national deployment was 
completed. 

Further, the department commented that Immigration and Customs 
Enforcement (ICE) had recently conducted enforcement operations at the 
Denver International Airport, and that the compliance rate during these 
operations increased from 30 percent to over 90 percent. It then concluded 
that the combined results of the exit pilot evaluation, the NSEERS pilot, 
and the ICE enforcement activities at the Denver International Airport lead 
it to believe that the US- VISIT exit alternatives have been adequately 
evaluated. 

We do not agree with this conclusion because it is based on unsupported 
assumptions. Specifically, DHS did not provide any evidence to support its 
claim that that US-VISIT would achieve a comparable compliance rate to 
the NSEERS program. Moreover, even if DHS could achieve a 75 percent 
compliance rate for US- VISIT exit, that still means that 25 percent of eligible 
persons would not be complying with the US- VISIT exit process. 

Further, DHS did not provide any information about the recent 
enforcement actions conducted by ICE, nor did it provide any evidence 
that this is a practical and viable option for the US-VISIT exit solution. 
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While we agree that enforcement actions may indeed increase the exit 
compliance rate, DHS has not yet assessed the impact of such a solution on 
the US-VISIT exit process. Further, the US-VISIT program director 
acknowledged the need to evaluate the impact of implementing potential 
enforcement actions on US-VISIT exit and planned to do so. 



We are sending copies of this report to the Chairmen and Ranking Minority 
Members of the Senate and House Appropriations Committees, as well as 
to the Chairmen and Ranking Minority Members of other Senate and House 
committees that have authorization and oversight responsibilities for 
homeland security. We are also sending copies to the Secretary of 
Homeland Security, Secretary of State, and the Director of OMB. Copies of 
this report will also be available at no charge on our Web site at 
www.gao.gov. 

Should you or your offices have any questions on matters discussed in this 
report, please contact me at (202) 512-3439 or at hiter@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs may be 
found on the last page of this report. Key contributors to this report are 
listed in appendix IV. 




Randolph C. Hite 

Director, Information Technology Architecture 
and Systems Issues 
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Chairman 

The Honorable Bennie G. Thompson 
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The Honorable Bob Filner 
House of Representatives 
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House of Representatives 

The Honorable Ruben Hinojosa 
House of Representatives 

The Honorable Solomon Ortiz 
House of Representatives 

The Honorable Silvestre Reyes 
House of Representatives 
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Our objective was to determine the progress of the Department of 
Homeland Security (DHS) in implementing 18 of our recommendations 
pertaining to the U.S. Visitor and Immigrant Status Indicator Technology 
(US- VISIT) program. To accomplish this objective, we reviewed and 
analyzed US-VISIT's most recent status reports on the implementation of 
our open recommendations and related key documents, augmented as 
appropriate by interviews with program officials. More specifically, we 
analyzed relevant systems acquisition documentation, including the 
program's process improvement plan, risk management plan, and 
configuration management plan. We also analyzed the US-VISIT security 
plan, privacy impact assessment, cost-benefit analysis, cost estimates, test 
plans, human capital plans, and related evaluations and assessments. In 
performing our analyses, we compared available documentation and 
program officials' statements with relevant federal guidance and associated 
best practices. 1 A more detailed description of our scope and methodology 
relative to the cost-benefit analysis, cost estimates, and test plans follows: 

• Our analysis of the cost-benefit analysis focused on Increment IB 
because this was the latest cost-benefit analysis and cost estimate 
prepared. In doing this analysis, we compared the US-VISIT cost-benefit 
analysis to eight criteria in Office of Management and Budget (OMB) 
guidance. 2 

• Our analysis of the cost estimate also focused on Increment IB for the 
same reason previously cited. In doing this analysis, we compared the 
estimate to 13 criteria from the Software Engineering Institute 3 that we 
have previously reported to be the minimum set of actions needed to 
develop a reliable cost estimate. We then determined whether the 
criteria were satisfied, partially satisfied, or not satisfied using the 
definitions given below. 



'See, for example, OMB, Guidance for Implementing the Privacy Provisions of the E- 
Government Act of 2002, OMB M-03-22 (Sept. 26, 2003); and Planning, Budgeting, 
Acquisition and Management of Capital Assets, Circular A-ll, Part 7 (Washington, D.C.: 
June 21, 2005). 

2 OMB, Planning, Budgeting, Acquisition and Management of Capital Assets, Circular A- 
11, Part 7 (Washington, D.C.: June 21, 2005) and Guidelines and Discount Rates for 
Benefits-Cost Analysis of Federal Programs, Circular A-94 (Washington, D.C.: Oct. 29, 
1992). 

3 Carnegie Mellon University Software Engineering Institute, A Manager's Checklist for 
Validating Software Cost and Schedule Estimates, CMU/SEI-95-SR-004 (January 1995). 
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• Our analysis of the test plans focused on Increment 2C because it is the 
most recently tested increment. This analysis included determining the 
extent to which the test plans for this increment met 4 key criteria that 
we have previously reported as essential to effective test plans. In doing 
this analysis, we examined Increment 2C systems documentation, 
including business and functional requirements and traceability 
matrixes. We also independently traced 58 business requirements and 
438 functional requirements to the test cases in the test plan. Further, 
we independently traced all test cases to the requirements to determine 
consistency. 

In performing our work, we used the following categories and definitions in 
deciding the extent to which each recommendation had been implemented. 
Specifically, we considered a recommendation 

• completely implemented when documentation demonstrated that it had 
been fully addressed, 

• partially implemented when documentation indicated that actions 
were under way to implement it, and 

• in progress when documentation indicated that action had been 
initiated to implement it. 

These categories and definitions are consistent with those used in our prior 
US-VISIT reports. 

In determining the amount of time it has taken to implement actions on our 
recommendations, we calculated the time from the date the report was 
issued through December 2005. 

We conducted our audit work at the US- VISIT program office in Rosslyn, 
Virginia, from August 2005 through December 2005, in accordance with 
generally accepted government auditing standards. 
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U.S. Department of Homeland Security 

Washington, DC 20528 



f$&$\ Homeland 
W Security 



January 13,2006 



Randolph C. Hite 

Director, Information Technology Architecture 
and Systems Issues 

U.S. Government Accountability Office 
Washington, D.C. 20548 



Dear Mr. Hite: 

Thank you for the opportunity to review the draft report, Homeland Security: Recommendations 
to Improve Management of Key Border Security Program Need to Be Implemented (GAO-06- 
296). As with prior reports that your office has issued regarding US- VISIT, there are many areas 
with which we agree, and the recommendations have made US-VISIT a stronger program. 
However, as with those past reports, the Department of Homeland Security (DHS) has certain 
areas of disagreement. They appear in our comments, which begin on page 2 of this letter. 

All of the issues covered by this report need to be viewed in the larger framework of one simple 
fact: US- VISIT is working as Congress intended. 

Thanks to the hard work and dedication of the US-VISIT team, all three congressionally 
mandated phases of implementation were completed ahead of schedule and under budget. US- 
VISIT is now in place at our nation's airports, seaports, and land border ports of entry. As you 
know, this program has a significant effect on our national security, economic prosperity, and 
international relationships around the world. Through biometric authentication, US- VISIT makes 
entering the U.S. easier for legitimate tourists, students, and business travelers, while making it 
more difficult to illegally enter and stay in our country. 

US- VISIT — working in partnership with stakeholders within DHS, the federal government, the 
private sector, and other countries — has exceeded the goals set by Congress and DHS for this 
program. In the final report of the 9/1 1 Commission, which issued grades to U.S. government 
responses to the recommendations outlined in its 2004 report, the 9/1 1 Commission awarded a 
"B" to "Biometric entry-exit screening system," one of the highest grades achieved by any 
government agency. The Commission recognized US-VISIT s successful screening operations at 
our ports of entry, and found that the program has collaborated well with Interpol. 



www.dhs.gov 
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In the two and a half years since its inception, US-VISIT has processed more than 45 million 
visitors at ports of entry, linking together systems from DHS and the Departments of State and 
Justice. In FY 2005, US-VISIT was successfully deployed at the 154 land border ports of entry 
(POEs), with the majority of ports reporting improved process times. US- VISIT also worked 
closely with the Department of State to implement the same capability at its 21 1 visa issuing 
posts around the world. US- VISIT has now intercepted nearly 1,000 prior or suspected criminals 
and immigration violators — including murderers, rapists, pedophiles, and drug traffickers — from 
entering the country, and enabled the Department of State to identify criminals and immigration 
violators who applied for visas. During this same period, DHS has provided 14,700 matches 
against the biometric watchlist to the Department of State through its Bio Visa program, which is 
fully integrated with US-VISIT. Use of biometrics has allowed the United States to deprive 
potential terrorists of one of the tools they use to threaten our nation and other countries around 
the world: the ability to cross our borders using fraudulent documents and violate our 
immigration laws without detection. 

Even with US-VISIT's increased security checks, travelers have not been inconvenienced; in 
fact, wait times at land border ports of entry have actually gone slightly down, and surveys from 
travelers show that the vast majority do not object to US-VISIT's biometric procedures. By 
working closely with federal, state, and local governments; conducting a thorough, concentrated, 
and continuing global outreach campaign; and through a commitment to respect for the privacy 
of those who would be enrolled in the system, US-VISIT has gained worldwide acceptance. US- 
VISIT's success inspired the European Union to adopt the inclusion of fingerprints into its 
biometric passports; and the government of Japan has indicated that it will model its own 
biometric border management system after US-VISIT. 

The GAO draft report is organized by discussion of progress on the implementation of prior open 
recommendations. US- VISIT comments on GAO's assessments are also provided by 
recommendation: 

Recommendation: 

Develop and begin implementing a system security plan, and perform a privacy impact 
assessment and use the results of the analysis in near-term and subsequent system acquisition 
decision. 

Response: 

While US-VISIT has completed a security plan and is in the process of completing a risk 
assessment, the relationship of these documents to system security must be clearly understood. 
As the GAO report details, US- VISIT is being implemented incrementally. Increments 1 through 
3 fulfilled legislative mandates through the introduction of interfaces and enhancements to 
existing "legacy" systems. As such, there is no US- VISIT system, but rather a US- VISIT 
program with capabilities delivered by these interconnected systems. Consistent with both 
National Institute of Standards (NIST) guidance and the DHS inventory, these systems have 
undergone extensive security evaluation leading to the certification and accreditation of each 
component system. The accreditation status of these systems is shown below: 
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System 


Status 


Expiration Date 


Arrival Departure Information System 
(ADIS) 


Authority to 
Operate 


11/12/06 


Treasury Enforcement Communications 
System (TECS) 


Authority to 
Operate 


02/06 

Renewal on 
schedule 


Automated Biometric Identification System 
(IDENT) 


Authority to 
Operate 


8/29/08 


Automated Identification Management 
System (AIDMS) 


Interim Authority to 
Operate 


1/26/06 

ATO on schedule 


Air/Sea Exit 


Interim Authority to 
Operate 


1/25/06 

ATO on schedule 



As an integral part of certification and accreditation, security plans and risk assessments are 
developed for each system. Additionally, risk mitigations are proposed and tracked in a DHS tool 
for each system. To posit that US-VISIT does not understand system requirements or did not 
ensure that "proper safeguards are in place to protect system data and resources" fails to 
acknowledge the extensive security procedures in place at the system level. 



As stated in the draft report, US-VISIT was preparing an enterprise-wide risk assessment. This 
document was completed in December 2005, and it identifies and proposes mitigations for 
security risks that arise from the complex interplay of the interconnected systems cited above. 
This document specifically addresses information security issues that might not be captured in 
the system-level documentation prepared for legacy system certification and accreditation. It also 
complements the security strategy document under development that supersedes the existing US- 
VISIT security plan. 

GAO properly notes that program management — as opposed to system security management — is 
the mechanism to address programmatic risks. US-VISIT coordinates issues derived from 
security reviews with a Risk Review Board to ensure that security issues are elevated when they 
impact overall program risk. 

In regard to the performance of privacy impact assessments, as GAO has noted, US- VISIT has 
completed numerous Privacy Impact Assessments (PIAs) and continues to update them to reflect 
changes in US-VISIT systems. The US-VISIT PIA is regarded throughout the privacy 
community as a model document. However, GAO appears to be unaware that the privacy 
program staff fully participates in US- VISIT integrated project teams and has effectively 
integrated privacy activities into the system development lifecycle by reviewing all system 
documents and performing privacy risk assessments for both specific issues as well as for overall 
increment planning and implementation. In this manner, US-VISIT believes that it has 
implemented the GAO recommendation to fully address privacy issues in the relevant system 
documentation, but understands that the privacy work completed was not always noted within 
each individual system document. To ensure that GAO has full visibility into the privacy work 
completed by US-VISIT in the future, all relevant system documents will be annotated to 
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specifically reference the privacy requirements and reference any privacy risk assessments that 
were completed. 

There are specific areas of the draft report's assessment of progress on this recommendation that 
need clarification: 

In the Executive Summary on page 17, first bullet, Security Plan: 

The US-VISIT Security Plan provided to GAO was composed in accordance with DHS 
requirements and NIST SP 800-18. The security plan devotes an entire section (section 4.1) to 
Risk Assessment and Management. In February 2005, GAO established another rinding to 
develop a program-wide risk assessment, which was completed at the end of calendar year 2005. 
This finding was only open for less than 10 months, not "about 30" as it appears in the chart... In 
addition to the program-wide risk assessment, US- VISIT certifies and accredits all of its systems 
in accordance with DHS policies and NIST 800-37 guidance. Systems that operate to achieve the 
US-VISIT mission have individual system-level risk assessments completed, evaluated, and 
updated throughout the lifecycle to ensure that risk is known and managed by US- VISIT 
program officials. These risk assessments have been provided to GAO. Plans of Actions and 
Milestones (POA&Ms) exist for each US-VISIT system— also provided to GAO— that establish 
an implementation schedule for mitigation strategies to reduce the overall risk to the systems. In 
addition to the system-level risk assessments and POA&Ms, risks determined to be significant to 
US- VISIT are elevated to the US-VISIT Risk Management Team. Based on all of the 
certification and accreditation efforts, existing system security risk assessments, and the program 
level risk management process, it is inaccurate to state that US- VISIT officials "are not in a 
position to know the risks associated with their program." 

In regard to Table 1, the length of time that GAO asserts that this recommendation has been open 
is inaccurate. The initial recommendation was to complete a US- VISIT Program Security Plan. 
The Security Plan was written in accordance with the format proscribed by NIST SP 800-18. It 
was delivered in September 2004, which should have closed the recommendation. A second 
follow-on recommendation from GAO to complete a program-level security risk assessment was 
issued in February 2005. US-VISIT is in the process of finalizing this document. 

In regard to the Privacy Impact Assessment, page 1 8: 

US-VISIT has completed numerous Privacy Impact Assessments (PIAs) and continues to update 
them to reflect changes in US- VISIT systems. The July 2005 PIA was found to be consistent 
with federal guidance, as stated in the draft report. That PIA was updated in December 2005 
based on the same guidelines. Numerous privacy risk assessments are also conducted to ensure 
that privacy is thoroughly accounted for throughout the entire US- VISIT program. The PIA has 
been updated to reflect all increments, and is considered to be part of system documentation. In 
addition, privacy is built into the US- VISIT lifecycle and is considered throughout the 
development of a system. GAO reports that privacy is not included in functional requirements 
documentation. A "functional privacy requirement" falls under the security controls and 
requirements which are included in both business and functional requirements documents. 
Security documentation specifically reflects that "Privacy Act Information" is processed by the 
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systems comprising US- VISIT. A FIPS 199 Security Categorization was performed for each 
system to determine that adequate security controls are in place or planned to protect this Privacy 
Act information. System Security Plans outline the specific controls in place to protect the data. 

Recommendation: 

Develop and implement a plan for satisfying key acquisition management controls, including 
acquisition planning, solicitation, requirements development and management, project 
management, contract tracking and oversight, evaluation, and transition to support, and 
implement the controls in accordance with the Software Engineering Institute's (SEI) guidance. 

Response: 

In regard to the discussion of the Capability Maturity Model-Integrated (CMMI): 

The draft report should reflect that, initially, US- VISIT adopted Carnegie Mellon University's 
Software Engineering Institute (SEI) Software Acquisition Capability Maturity Model® (SA- 
CMM®) to guide its management process implementation. US-VISIT transitioned from the SA- 
CMM to the Capability Maturity Model-Integration (CMMI®) in December 2004 based on 
recommendations from the SEI, MITRE, and the newly hired US- VISIT Process Improvement 
Lead. The CMMI® is a more robust model and is now the "best practice" standard in use at 
hundreds of commercial and government organizations. Additionally, SEI expects to retire the 
SA-CMM® very soon. SEI developed a guidance document — the CMMKS-Acquisition 
Module — to assist acquisition organizations such as US-VISIT in applying the CMMI®. As a 
result, the US-VISIT process improvement strategy and plans, process development, and 
appraisals are now realigned to the selected CMMI® process areas most applicable to US-VISIT. 

Recommendation: 

Clarify the operational context in which US-VISIT is to operate. 
Response: 

As noted in the draft report, ". . .an immigration and border management strategic plan was 
drafted in March 2005 that shows how US-VISIT is aligned with DHS' organizational mission 
and defines an overall vision for immigration and border management." GAO further noted that, 
"Since the plan was drafted, DHS has reported that other relevant initiatives have been 
undertaken, such as the Security and Prosperity Partnership of North America and the Secure 
Border Initiative." And the draft report concluded that, "Until US-VISIT's operational context is 
fully defined, DHS is increasing its risk of defining, establishing, and implementing a program 
that is duplicative of other programs and not interoperable with them." 

The mission of DHS is continually expanding and, as a result, the scope of US-VISIT's activities 
in providing for capabilities to meet that mission is constantly evolving. US- VIS IT agrees that 
the operational context in which it operates is, in a sense, "in progress" in that it continues to 
evolve in compliance with new legislative, administrative, and Departmental mandates and 
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priorities. However, the principal role of US-VISIT is to integrate information and make 
interoperable immigration and border management systems across the Departments of Homeland 
Security and State and, as such, US- VISIT will be an enabler of other programs. A significant 
part of US-VISIT s role is to establish an environment that will ensure agencies work toward a 
common environment that will eliminate redundancies. The immigration and border 
management strategic plan, as well as the first MCE derived from that plan, are being used in 
current operations. Elements of this plan are being incorporated into the planning and operational 
context for the projects noted by GAO as having potential for redundancy. Although US-VISIT 
concurs that more certainty would be desirable, mechanisms to mitigate the risk noted by GAO 
have been developed and are being implemented. 

Recommendation: 

Determine whether proposed US-VISIT increments will produce mission value commensurate 
with cost and risks and disclose to the Congress planned actions. 

Response: 

US-VISIT disagrees with the assertion in the draft report that it did not perform a complete 
uncertainty analysis for the three alternatives. A comprehensive uncertainty analysis was 
conducted throughout the study. The Risk Analysis Process, summarized in Appendix F, is a 
state-of-the-art process to account for uncertainty surrounding key benefit and cost assumptions 
used in the analysis. Chapter 6 of the cost benefit analysis (CBA) explicitly shows the 
assumptions used in the analysis, expressed in the form of ranges built around the major 
variables. These assumptions are based on observations of historical trends, pilot study results, 
and expert opinion solicited during risk analysis sessions that were organized with the 
participation of various stakeholders. Therefore, the process incorporates both objective and 
subjective perspectives. The results of the risk analysis are subsequently portrayed as 
probabilistic distributions in Chapter 7. This approach is comprehensive, more rigorous, and 
more appropriate for this study than sensitivity analysis. Sensitivity analysis theoretically 
provides insight into which factors in the decision are most important. Risk analysis, on the hand, 
allows for the simultaneous variation of key assumptions within their assigned boundaries — a 
better reflection of reality — rather than varying one variable at a time. The risk analysis outcome 
is more appropriate for this study as the results must provide the decision maker with an 
indication of the project's worthiness given the existing uncertainty, rather than how the outcome 
is sensitive to one specific variable or another. 

US- VISIT was guided by, and adhered to, OMB Circular A-94 and the DHS CBA handbook, 
Capital Planning and Investment Control: Department of Homeland Security Cost Benefit 
Analysis (CBA) Work Book, May 2003, in developing the Increment IB CBA. US- VISIT'S 
disagreement fundamentally concerns expectations as to the scope and level of detail of analysis 
that should be included with the formal CBA document. The auditors apparently believe that all 
detail should be included within the formal CBA document. US- VISIT instead chose to 
communicate the substance of its analysis in the formal CBA, believing the results of the final 
analyses were the more relevant input for DHS decision-makers. US-VISIT's reading of Circular 
A-94 and the DHS CBA Work Book does not lead to the conclusion that these documents 
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require the level of detail GAO desires. US-VISIT provided GAO with some of the detailed 
analyses supporting the Increment IB CBA, and is prepared to provide other detailed analyses 
for GAO review. 

US-VISIT also takes exception to GAO's assertions in Table 2: US-VISIT Satisfaction of OMB 
Economic Analysis Criteria. For Criterion 5, "The quality of the benefits to be realized from 
each alternative was reasonable," GAO concludes that the criterion was not met based upon its 
analysis that "Year-by-year benefit estimates were not reported." It is important to note that the 
net present value (NPV) estimate was based upon an estimation of the stream of benefits and 
costs annually. The NPV cannot be estimated without a year-by-year benefit analysis. The 
detailed annual analysis GAO desires was performed and is available for review. Again, the 
content of the formal CBA was focused on meeting the information needs of DHS executives, 
with detailed supporting analyses available upon request. For Criterion 8, "a complete 
uncertainty analysis of cost and benefit was included," GAO concludes that the criterion was not 
met based upon its analysis that "Although the cost-benefit analysis did include Monte Carlo 
simulation results for the three exit alternatives, no sensitivity analysis was conducted for those 
alternatives. Instead, the cost-benefit analysis reports sensitivity analysis results for the five 
deployment scenarios." US- VISIT disagrees with the assertion that it did not perform a complete 
uncertainty analysis for the three alternatives. A comprehensive uncertainty analysis was 
conducted. 

The draft report also states, "It is important that the program adhere to relevant guidance in 
developing its incremental cost-benefit analyses. If this is not done, the reliability of the analyses 
is diminished, and an adequate basis for the prudent investment decision-making does not exist. 
Moreover, if the mission value of a proposed investment is not commensurate with costs, it is 
vital that this information be fully disclosed to DHS and congressional decision makers. The 
underlying intent of our recommendation is that this information be available to inform such 
decisions." US-VISIT believes that the Increment IB CBA does conform to relevant guidance 
and that the heart of the disagreement with GAO involves a difference in interpretation as to the 
amount of detail necessary for inclusion within the formal CBA, as opposed to having supporting 
detailed analyses available upon request. Further, the NPV of each Increment IB alternative was 
clearly communicated in the executive summary of the CBA in order to provide decision makers 
with the primary measure of each alternative's relative worthiness. As these NPVs indicate, any 
investment with a five-year lifecycle and considered interim in nature will face a considerable 
challenge in providing economic benefits commensurate with cost. To quote the CBA, "The full 
economic benefit of this exit solution is not realized during the initial five years of operation, but 
is harvested over an adequate life cycle of the investment." 

Recommendation : 

Develop and implement a risk management plan and ensure that all high risks and their status 
are reported regularly to the executive body. 

Response: 

In analyzing US-VISIT's efforts at managing risk, it is important to consider that US- VISIT 
began the development and implementation of its risk management plan in 2004 immediately 
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after GAO made its initial recommendation. As part of its CMMI process maturity baseline 
internal appraisal completed in July 2005, US- VISIT found that the risk management process 
detailed in its plan was not consistently applied across the program. In response, positive steps 
have since been taken. The Risk Management Plan was approved in September 2005 and 
includes, among other things, a process for planning, identifying, analyzing, handling, and 
monitoring risk. It also defines the governance structure to be used in overseeing and managing 
the process. US- VISIT also maintains a risk management database, which includes among other 
things a description of the risk, its priority (high, medium, or low) and impact, and its mitigation 
strategy. The database is currently available to program management and staff. 

US-VISIT established a Risk Review Board, Risk Review Council, and Risk Owner to govern its 
risk activities. The roles and responsibilities are described below. 

• The Risk Review Board directs all risk governance within the program and provides the 
mechanism to escalate/transfer the consideration of risks to program governing boards 
and to organizations external to the program. 

• The Risk Review Council oversees and manages risks that are significant, controversial, 
or cross-project, or that may require escalation to the Risk Review Board. 

• Risk Owners analyze, handle, and monitor risks. 

Risk management training has been developed and training sessions for US-VISIT personnel and 
contractors began in November 2005. The Risk Review Board, chartered in September 2004, 
reviews risks with US- VISIT executives and has been meeting periodically since January 2005. 

Recommendation: 

Develop and approve test plans before testing begins that (1) specify the test environment; (2) 
describe each test to be performed, including test controls, inputs, and expected outcomes; (3) 
define the test procedures to be followed in conducting the tests; and (4) provide traceability 
between test cases and the requirements to be verified by the testing. 

Response: 

While there were minor issues with the traceability of requirements to test cases, the extent of the 
discrepancies is far less than presented by the draft report. The data cited in the report is 
consistent with GAO's initial findings as reported in its document, Topics for Discussion and 
Request for Documentation Regarding Testing of US- VISIT Increment 2C Proof of Concept 
Phase I, received on October 12, 2005, by US-VISIT. However, the findings do not accurately 
reflect the status of Increment 2C Phase 1 testing. 

In the October 12, 2005, document, GAO requested the updated version of the Requirements 
Traceability Matrix (RTM) to ". . .show proof that the test cases were actually executed and the 
outcome(s) achieved." GAO also requested the updated RTM to resolve requirements and test 
case mapping issues identified in the GAO report. US-VISIT System Assurance provided the 
current versions of the US-VISIT Increment 2C RTM along with current versions of the US- 
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VISIT Increment 2C Test Plan on November 9, 2005, to GAO. Documents provided that day 
included: 

• US-VISIT Increment 2C Requirements Traceability Matrix 

• US-VISIT Increment 2C Proof of Concept IV&V Test Cases 

• US-VISIT Increment 2C Proof of Concept IV&V Test Cases Appendix A - H 

• US-VISIT System Engineering Plan 

• US-VISIT Task Order 4 Option Year 1 

These documents resolved the issues that GAO identified with earlier versions of the documents, 
namely test case traceability to requirements and testing results. 

Recommendation: 

Implement effective configuration practices, including establishing a US-VISIT change control 
board to manage and oversee system changes. 

Response: 

The draft report states that ". . .changes to component systems that are initiated and approved by 
another DHS organization and that could affect US- VISIT performance are not subject to US- 
VISIT configuration management processes and are not also being examined and approved by 
the US- VISIT control board. This lack of US-VISIT control was the impetus for our 
recommendation." A representative from US- VISIT'S Office of Mission Operations or Office of 
Information Technology attends all CCB meetings for applicable legacy component systems. 
Any proposed change request from a legacy component CCB that could affect US-VISIT 
functionality is brought by the US- VISIT representative to the US-VISIT ECCB for 
consideration. 

Recommendation: 

Assess the full impact of Increment 2B on land POE workforce levels and facilities, including 
performing appropriate modeling exercises. 

Response: 

The draft report asserts that the scope of US-VISIT's evaluation of the impact of Increment 2B 
was too limited. Given the imperative to meet the December 31, 2004, legislative mandate, US- 
VISIT's Increment 2B was limited by time, funding, and resources, and as such the performance 
evaluation had to focus on representative sites. Three pilot sites were identified by Customs and 
Border Protection (CBP), and the selection criteria were based upon input from US- VISIT as 
well as CBP's own operational constraints. The three locations offered by CBP provided a 
reasonable mix of travelers and did not have other constraints that would directly impact the 
collection of performance data specific to the Form 1-94 issuance. 
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Wait times are a complex function of CBP operations, receipt of intelligence, traffic volume, 
staffing levels, availability of Officers to staff lanes/booths, weather, seasonal changes to traffic, 
holidays, and local events. Since Increment 2B incorporated the collection of a biometric into 
the previously manual process of Form 1-94 issuance, which is only one process in CBP border 
operations, measurements were taken that specifically addressed the delta introduced by 
Increment 2B. [In addition, on page 38, Table 3, concerning the reduction in reported processing 
times, has an incorrect heading for the last column: it should read "(February 2005)," not 
"(February 2004)."] 

Going back to assess the full impact of Increment 2B would require baseline data collection that 
represents operational performance prior to the Increment 2B deployment. This is not practicable 
in the production environment that exists at the 47 ports that were not evaluated. The alternative 
approach is to model the baseline performance using historical data from the three ports 
evaluated and possibly supplement this data with data from previous studies. However, it is very 
likely that the modeling approach used to reconstruct the baseline performance will be subject to 
question. The detailed step-by-step processing times are site specific and not easily generalized 
from one port to another. As a result, any baseline estimates prepared ex post will not be as 
accurate as the actual results reported from the three ports. Lacking an acceptable baseline, any 
conclusions developed from such a follow-up study on the remaining 47 ports could be refuted. 

The reference in the draft report to the number of workstations (baseline versus evaluation) is 
confusing. The number of workstations available to process applicants for a Form 1-94 and/or the 
number of Officers available to operate those workstations are often utilized to address the 
number of applicants (or volume). Such resources do not impact the time it takes to issue a Form 
1-94 to an individual; consequently, the time it takes to issue a Form 1-94 is the only true valid 
measure. 

The draft report also describes the San Ysidro port of entry (POE) as the busiest land POE. This 
is not entirely accurate; while San Ysidro is the largest POE by volume of travelers, the three 
bridges combined for Laredo make it the busiest port that issues Form I-94s. In 2003, San Ysidro 
issued approximately 409,683 I-94s; the combined bridges at Laredo issued 432,892 Form I-94s. 

Recommendation: 

Develop a plan, including explicit tasks and milestones, for implementing all our open 
recommendations and periodically report to the DHS Secretary and Under Secretary on 
progress in implementing this plan; and report this progress, including reasons for delays, in all 
future expenditure plans. 

Response: 

GAO's assertion that 19 months elapsed from the issuance of this recommendation until US- 
VISIT assigned responsibilities to specific individuals for addressing each recommendation is 
untrue. In fact, the first such plan for addressing GAO recommendations was issued on August 
1 8, 2003— less than a month after former DHS Secretary Ridge officially created the US-VISIT 
program office. Subsequent reports, issued periodically and updated with progress on 
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implementation, have included all additional recommendations as they appeared in all GAO 
reports affecting US-VISIT. 

Recommendation: 

Follow effective practices for estimating the costs of future increments. 
Response: 

US-VISIT disagrees with GAO's evaluation in Table 4 of the Increment IB cost benefit analysis 
against the 13 SEI criteria for satisfaction of cost estimating. 

For Criterion 2, the lifecycle to which the estimate applies is clearly defined. GAO concludes 
that the criterion was partially met based upon its analysis that "The lifecycle was not clearly 
defined to ensure that the full cost of the program was included. For example, the analysis did 
not include evidence that nonrecurring development costs were included in the cost estimate." 
US- VISIT does agree that it did not clearly identify the lifecycle to which the estimate applies. 
The crux of the disagreement is once again related to the purpose of the CBA document, which is 
to inform DHS decision makers as to the relative worthiness of each of the three exit alternatives 
considered for deployment as part of Increment IB. The analysis supports the decision related to 
the deployment of an operational solution for the project. It does not analyze conceptual 
alternatives early in the investment lifecycle that would necessitate the inclusion of planning, 
analysis, design, and development activities in the cost estimates for each alternative, as these 
activities had already occurred and therefore had no bearing on the decision to deploy. The 
general cost assumptions listed in Chapter 6 of the CBA include the following lifecycle 
assumption: "Cost estimates represent only the incremental cost associated with acquiring and 
maintaining the interim exit solution to be delivered to 76 airports and 12 seaports as part of 
Increment IB." Within the context of that overall lifecycle assumption, the following 
information technology cost assumption is stated in the CBA: "IT systems development, 
integration, and security costs [are] assumed to be sunk historical costs incurred prior to full 
deployment of exit alternatives and therefore not included in cost estimates." In other words, the 
analysis includes only those acquisition costs that will be incurred as a result of the decision on 
which exit alternative to deploy, and does not include sunk costs for the plan, analyze, design, 
build, and test stages that have already been incurred and do not impact the deployment decision 
informed by this analysis. Per the DHS CBA Work Book, pages 33-34, "Sunk costs are not 
relevant to the current investment analysis because only current decisions can affect the future 
consequences of investment alternatives. The IPT will not include sunk costs in any CBA 
calculations." 

For Criterion 3, "The task has been appropriately sized," GAO concludes that the criterion was 
not met based upon its analysis that "An appropriate sizing metric should be used in the 
development of the estimate, such as the amount of software to be developed and the amount of 
software to be revised. The program office provided no evidence that an appropriate sizing 
mechanism was used, and program officials stated that they had not collected these data." US- 
VISIT believes that it appropriately sized the task described in the cost estimates for the 
Increment IB Exit CBA alternatives. As stated above, the alternatives considered in the analysis 
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represent operational deployment alternatives, not conceptual program initiation phase 
alternatives. Therefore, activities related to the plan, analyze, design, build, and test stages were 
not considered relevant to the scope of the estimates and were not included. Sizing metrics 
related to software development were not applicable to the deployment phase because these 
activities had already occurred and were therefore considered sunk costs not to be included in the 
CBA calculations. Sizing metrics relevant to the deployment phase were used in the cost 
estimates and were derived based upon the actual costs of deployment experienced during the 
exit pilot. By determining the average cost of deployment for sample airports and a seaport based 
upon size and relative activity, and extrapolating those sample deployment cost estimates across 
their respective operational environments, a total cost of deployment was calculated. The 
deployment cost estimate sizing technique described above is clearly communicated in the CBA 
in the general cost assumptions in Chapter 6. 

For Criterion 5, "A written summary of parameter values and their rationales accompanies the 
estimate," GAO concludes that the criterion was partially met based upon its analysis that "If a 
parametric equation was used to generate the estimate, the parameters that feed the equation 
should be provided along with an explanation of why they were chosen. High-level cost 
categories, such as labor, information technology, facilities, and other costs were identified, but 
detailed parameters used to develop the estimate, such as number of software lines of code, were 
not provided in the analysis." US- VIS IT did provide the detailed parameters used to develop the 
cost estimates for the Increment IB Exit CBA alternatives. As stated above, the alternatives 
considered in the analysis represent operational deployment alternatives, not conceptual program 
initiation phase alternatives. Therefore activities related to the plan, analyze, design, develop, 
and test stages were not considered relevant to the scope of the estimates and were not included. 
Parameters related to software development, such as the number of software lines of code, were 
not applicable to the deployment phase because these activities had already occurred and were 
therefore considered sunk costs not to be included in the CBA calculations. Cost estimating 
parameters relevant to the deployment phase were used in the cost estimates and were derived 
from actual costs of deployment experienced during the exit pilot. By deterrnining the average 
cost of deployment for sample airports and a seaport based upon size and relative activity, and 
extrapolating those sample deployment cost estimates across their respective operational 
environments, a total cost of deployment was calculated. The deployment cost estimating 
parameters described above are clearly communicated in the CBA in the general cost 
assumptions in Chapter 6. 

For Criterion 7, "A structured process, such as a template or format, has been used to ensure that 
key factors have not been overlooked," GAO concluded that the criterion was partially met based 
upon its analysis that "The analysis included four high-level cost categories (labor, facilities, 
operations and maintenance, and information technology), but did not include a detailed work 
breakdown structure and omitted important cost elements, such as system testing and training." 
US- VISIT agrees that the estimate was not derived using a work breakdown structure, although 
it did use the available project implementation schedule as a proxy for the activities related to the 
deployment of the Increment IB exit criterion. However, US-VISIT disagrees with GAO's 
assertion that the cost categories did not include important cost elements such as system testing 
and training. The analysis examined the costs of labor, facilities, operations and maintenance, 
information technology, travel, and training as stated in Chapter 6 of the CBA. In addition, as 



Page 75 



GAO-06-296 US-VISIT Recommendations 



Appendix II 

Comments from the Department of Homeland 
Security 



13 

stated above, the alternatives considered in the analysis represent operational deployment 
alternatives, not conceptual program initiation phase alternatives. Therefore, activities related to 
the plan, analyze, design, build, and test stages were not considered relevant to the scope of the 
estimates and were not included. Costs related to systems development and testing were not 
applicable to the deployment phase because these activities had already occurred and were 
therefore considered sunk costs not to be included in the CBA calculations. 

For Criterion 8, "Uncertainties in parameter values have been identified and quantified," GAO 
concludes that the criterion was partially met based upon its analysis that "A sensitivity and risk 
analysis was performed, but this analysis did not identify detailed parameter values." As stated 
previously, US- VISIT did conduct a comprehensive uncertainty analysis. 

Recommendation : 

Reassess plans for deploying an exit capability to ensure that the scope of the exit pilot provides 
for adequate evaluation of alternative solutions and better ensures that the exit solution selected 
is in the best interest of the program. 

Response: 

The draft report states that ". . .questions remain about whether the exit alternatives have been 
evaluated sufficiently to permit selection of the best exit solution for national deployment." The 
draft report raises questions about the effectiveness of the three alternatives since the average 
compliance rate was only 24 percent for the three alternatives. 

The GAO analysis fails to take into account the compliance rate of the previous pilot program for 
exit, the National Security Entry Exit Registration System (NSEERS). Since its inception, the 
NSEERS compliance rate is 75 percent. NSEERS has very limited exit locations — typically not 
in the departure areas of airports — for aliens to biometrically check out. Therefore, any of the 
three alternatives tested would have at least a minimum 75 percent compliance rate once the 
national deployment was completed. This information was not in the evaluation report but was 
presented in the US-VISIT memorandum to the Deputy Secretary with the subject, Direction for 
the US-VISIT Air/Sea Exit Program. 

GAO also states that the effect of the enforcement mechanism to improve compliance is 
unknown and that additional evaluation is warranted. However, within the past two months, 
Immigration and Customs Enforcement (ICE) has conducted enforcement operations at the 
Denver International Airport. As a result of these enforcement efforts, the compliance rate at 
Denver International Airport has increased from 30 percent to over 90 percent. The combined 
results of the US- VISIT exit evaluation, the NSEERS pilot, and the ICE enforcement activities at 
Denver International Airport lead US- VISIT to believe that the exit alternatives have been 
adequately evaluated. 



While we may disagree with some of GAO's assessment of the amount of progress on the open 
recommendations addressed in the draft report, we nevertheless concur in the need for their 
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implementation with all due speed and diligence. However, in perspective, the discussion of 
these recommendations does not alter the overall assessment of the Department — and many 
others — that US-VISIT's continuing success is making a valuable contribution to the enhanced 
security of the United States. 

Sincerely, 

Steven J. Pecinovsky 

Director, Departmental GAO/IG Liaison Office 
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US-VISIT involves complex processes governing the stages of a traveler's 
visit to the United States (pre-entry, entry, status, and exit) and analysis of 
hundreds of millions of foreign national travelers at over 300 air, sea, and 
land ports of entry (POE). A simplified depiction of these processes is 
shown in figure 4. 



Figure 4: US-VISIT Process Overview 




Sources: US-VISIT, GAO (analysis), Nova Development Corp. (images). 
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Pre-entry Process 



Pre-entry processing begins with initial petitions for visas, grants of visa 
status, or the issuance of travel documentation. When a foreign national 
applies for a visa at a U.S. consulate, biographic and biometric data are 
collected and shared with border management agencies. The biometric 
data are transmitted from the Department of State to DHS, where the prints 
are run against the Automated Biometric Identification System (IDENT) 
database 1 to verify identity and to run a check against the biometric watch 
list. The results of the biometric check are transmitted back to State. A "hit" 
response prevents State's system from printing a visa for the applicant until 
the information is reviewed and cleared by a consular officer. 



Pre-entry also includes transmission by commercial air and sea carriers of 
crew and passenger manifests to appropriate immigration officers before 
these carriers arrive in the United States. 2 These manifests are transmitted 
through the Advanced Passenger Information System (APIS). The APIS 
lists are run against the biographic lookout system to identify those arrivals 
for whom biometric data are available. In addition, POEs review the APIS 
list in order to identify foreign nationals who need to be scrutinized more 
closely. 



Entry PrOCeSS When a foreign national arrives at a POE's primary (air and sea) or 

secondary (land) inspection booth, the inspector, using a document reader, 
scans the machine-readable travel documents. APIS returns any existing 
records on the foreign national to the US- VISIT workstation screen, 
including manifest data matches and biographic lookout hits. When a 
match is found in the manifest data, the foreign national's name is 
highlighted and outlined on the manifest data portion of the screen. 

Biographic information, such as name and date of birth, is displayed on the 
bottom half of the computer screen, along with a photograph obtained 



'IDENT collects and stores biometric data about foreign nationals, including Federal Bureau 
of Investigation information on all known and suspected terrorists, selected wanted persons 
(foreign-born, unknown place of birth, previously arrested by DHS), and previous criminal 
histories for high-risk countries; DHS Immigration and Customs Enforcement information 
on deported felons and sexual registrants; and DHS information on previous criminal 
histories and previous IDENT enrollments. Information from the FBI includes fingerprints 
from the Integrated Automated Fingerprint Identification System. 

Enhanced Border Security and Visa Entry Reform Act of 2002, Pub. L. No. 107-173 (May 14, 
2002). 
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from State's Consular Consolidated Database. 3 The inspector at the booth 
scans the foreign national's fingerprints (left and right index fingers) and 
takes a digital photograph. This information is forwarded to the IDENT 
database, where it is checked against stored fingerprints in the IDENT 
lookout database. If the foreign national's fingerprints are already in 
IDENT, the system performs a match (a comparison of the fingerprint 
taken during the primary inspection to the one on file) to confirm that the 
person submitting the fingerprints is the person on file. If no prints are 
currently in IDENT, the foreign national is enrolled in US-VISIT (i.e., 
biographic and biometric data are entered into IDENT). 

During this process, the inspector also questions the foreign national about 
the purpose of his or her travel and length of stay. The inspector adds the 
class of admission and duration of stay information into the Treasury 
Enforcement Communications Systems, 4 and stamps the "admit until" date 
on the Form I-94. 5 If the foreign national is ultimately determined to be 
inadmissible, the person is detained, lookouts are posted in the databases, 
and appropriate actions are taken. 



StcltllS Mcina.gGni.Gnt ^ ne s t a t us management process manages the foreign national's temporary 

p " presence in the United States, including the adjudication of benefits 

UCtibb applications and investigations into possible violations of immigration 

regulations. 



3 The Consular Consolidated Database is a system that includes information on whether a 
visa applicant has previously applied for a visa or currently has a valid visa. 

4 Treasury Enforcement Communications Systems maintains lookout data and interfaces 
with other agencies' databases; it is currently used by inspectors at POEs to verify traveler 
information and update traveler data. 

5 The Form 1-94 is used to track the arrival and departure of nonimmigrants. It is divided into 
two parts. The first part is an arrival portion, which includes, for example, the 
nonimmigrant's name, date of birth, and passport number. The second part is a departure 
portion, which includes the name, date of birth, and country of citizenship. 
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As part of this process, commercial air and sea carriers transmit departure 
manifests electronically for each departing passenger. These manifests are 
transmitted through APIS and shared with the Arrival Departure 
Information System (ADIS). 6 ADIS matches entry and exit manifest data 
(i.e., each record showing a foreign national entering the United States is 
matched with a record showing the foreign national exiting the United 
States). ADIS also receives status information from the Computer Linked 
Application Information Management System 7 and the Student Exchange 
Visitor Information System 8 on foreign nationals. 



Exit PrOCGSS ^ ne ex ^ P rocess includes the carriers' submission of electronic manifest 

data to APIS. This biographic information is transmitted to ADIS, where it 
is matched against entry information. At the 11 POEs where the exit 
solution is being implemented, the departure is processed by one of three 
exit methods. Within each port, one or more of the exit methods may be 
used. The three methods are as follows: 



• Kiosk: At the kiosk, the traveler, guided by a workstation attendant if 
needed, scans the machine-readable travel documents, provides 
electronic fingerprints, and has a digital photograph taken. A receipt is 
printed to provide documentation of compliance with the exit process 
and to assist in compliance on the traveler's next attempted entry to the 
country. After the receipt prints, the traveler proceeds to his or her 
departure gate. At the conclusion of the transaction, the collected 
information is transmitted to IDENT. 



• Mobile device: At the departure gate, and just before the traveler boards 
the departure craft, either a workstation attendant or law enforcement 
officer scans the machine-readable travel documents, scans the 
traveler's fingerprints (right and left index fingers), and takes a digital 
photograph. A receipt is printed to provide documentation of 



6 ADIS is a database that stores traveler arrival and departure data and that provides query 
and reporting functions. 

7 The Computer Linked Application Information Management System is a system that 
contains information on foreign nationals who request benefits, such as change of status or 
extension of stay. 

8 The Student Exchange Visitor Information System is a system that contains information on 
foreign students. 
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compliance with the exit process and to assist in compliance on the 
traveler's next attempted entry to the country. The device wirelessly 
transmits the captured data in real time to IDENT via the Transportation 
Security Administration's Data Operations Center. 

If the device is being operated by a workstation attendant, he or she 
provides a printed receipt to the traveler, and the traveler then boards 
the departure craft. If the mobile device is being operated by a law 
enforcement officer, the captured biographic and biometric 
information is checked in near real time against watch lists. Any 
potential match is returned to the device and displayed visually for the 
officer. If no match is found, the traveler is allowed to board the 
departure craft. 

• Validator: Using a kiosk, the traveler, guided by a workstation attendant 
if needed, scans the machine-readable travel documents, provides 
electronic fingerprints, and has a digital photograph taken. 

As with the kiosk, a receipt is printed to provide documentation of 
compliance with the exit process and to assist in compliance on the 
traveler's next attempted entry to the country. However, this receipt has 
biometrics (i.e., the traveler's fingerprints and photograph) embedded 
on the receipt. At the conclusion of the transaction, the collected 
information is transmitted to IDENT. 

The traveler presents his or her receipt to the attendant or law 
enforcement officer at the gate or departure area, who scans the receipt 
using a mobile device. The traveler's identity is verified against the 
biometric data embedded on the receipt. Once the traveler's identity is 
verified, he or she is allowed to board the departure craft. The captured 
data are not transmitted in real time back to IDENT. Instead, the data 
are periodically uploaded through the kiosk to IDENT. 



Analysis PrOCGSS ^^y^s capability is to provide for the continuous screening against 

watch lists of individuals enrolled in US-VISIT for appropriate reporting 
and action. As more entry and exit information becomes available, it is to 
be used for analysis of traffic volume and patterns as well as for risk 
assessments. The analysis is also to be used to support resource and 
staffing projections across POEs, strategic planning for integrated border 
management analysis performed by the intelligence community, and 
determination of travel use levels and expedited traveler programs. 
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